[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:48 dist-test-slave-csg5 krb5kdc[32377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:48 dist-test-slave-csg5 krb5kdc[32377](info): set up 2 sockets
Aug 12 07:07:48 dist-test-slave-csg5 krb5kdc[32377](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 07:07:49.826421 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:07:49 dist-test-slave-csg5 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:07:49.992293 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:34069
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:34069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:50.125021 32393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:50.125373 32393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:50.125418 32393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:50.129691 32393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:50.129842 32393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:50.129884 32393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:50.129928 32393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:50.129961 32393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:50.135694 32393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:34069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:34069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:50.136976 32393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:50.138084 32393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:07:50.149138 32393 server_base.cc:1034] running on GCE node
W20260812 07:07:50.149165 32400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:50.149170 32402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:50.149201 32399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:50.150214 32393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:50.151841 32393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:50.153122 32393 hybrid_clock.cc:648] HybridClock initialized: now 1786518470153118 us; error 75 us; skew 500 ppm
Aug 12 07:07:50 dist-test-slave-csg5 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518470, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:50.157061 32393 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:07:50.158177 32393 webserver.cc:460] Webserver started at http://127.31.156.126:35177/ using document root <none> and password file <none>
I20260812 07:07:50.158950 32393 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:50.159024 32393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:50.159325 32393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:50.161849 32393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "765204bbe6b64a60a91af8628275fd6d"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-csg5"
server_key: "4cf51a5f64ecf1d34dec24fd91c0471f"
server_key_iv: "9f7cff18ec2c570ca65545fb3808480f"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.162711 32393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "765204bbe6b64a60a91af8628275fd6d"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-csg5"
server_key: "4cf51a5f64ecf1d34dec24fd91c0471f"
server_key_iv: "9f7cff18ec2c570ca65545fb3808480f"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.167776 32393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:07:50.171082 32412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:50.172863 32393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:07:50.173010 32393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "765204bbe6b64a60a91af8628275fd6d"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-csg5"
server_key: "4cf51a5f64ecf1d34dec24fd91c0471f"
server_key_iv: "9f7cff18ec2c570ca65545fb3808480f"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.173309 32393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:50.192633 32393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:50.197244 32393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:50.197532 32393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:50.207290 32471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:34069 every 8 connection(s)
I20260812 07:07:50.207433 32393 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:34069
I20260812 07:07:50.208545 32393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:50.212023 32472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:50.212474 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 32393
I20260812 07:07:50.212706 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:07:50.214553 32369 external_mini_cluster.cc:1428] Setting key 66df30754ec6dbf967c60ed7bbea6d35
I20260812 07:07:50.220746 32472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d: Bootstrap starting.
I20260812 07:07:50.224093 32472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:50.225080 32472 log.cc:824] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:50.228075 32472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d: No bootstrap required, opened a new log
I20260812 07:07:50.233130 32472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765204bbe6b64a60a91af8628275fd6d" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34069 } }
I20260812 07:07:50.233558 32472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:50.233656 32472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 765204bbe6b64a60a91af8628275fd6d, State: Initialized, Role: FOLLOWER
I20260812 07:07:50.234216 32472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765204bbe6b64a60a91af8628275fd6d" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34069 } }
I20260812 07:07:50.234411 32472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:50.234520 32472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:50.234695 32472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:50.235881 32472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765204bbe6b64a60a91af8628275fd6d" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34069 } }
I20260812 07:07:50.236331 32472 leader_election.cc:302] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 765204bbe6b64a60a91af8628275fd6d; no voters:
I20260812 07:07:50.236752 32472 leader_election.cc:288] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:50.236863 32478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:50.237097 32478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [term 1 LEADER]: Becoming Leader. State: Replica: 765204bbe6b64a60a91af8628275fd6d, State: Running, Role: LEADER
Aug 12 07:07:50 dist-test-slave-csg5 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:50.237454 32478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765204bbe6b64a60a91af8628275fd6d" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34069 } }
I20260812 07:07:50.238302 32472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:50.239835 32479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "765204bbe6b64a60a91af8628275fd6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765204bbe6b64a60a91af8628275fd6d" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34069 } } }
I20260812 07:07:50.240042 32479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:50.240520 32489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:50.239830 32480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 765204bbe6b64a60a91af8628275fd6d. Latest consensus state: current_term: 1 leader_uuid: "765204bbe6b64a60a91af8628275fd6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765204bbe6b64a60a91af8628275fd6d" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34069 } } }
I20260812 07:07:50.240630 32480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:50.244781 32489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:50.249191 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:50.218728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44286 (local address 127.31.156.126:34069)
0812 07:07:50.219325 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:50.219344 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:50.220849 (+ 1505us) server_negotiation.cc:407] Connection header received
0812 07:07:50.225078 (+ 4229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:50.225098 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:50.225671 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:50.226096 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:50.229023 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.230237 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.231882 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.232446 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.232671 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:50.238493 (+ 5822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:50.238502 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:50.238535 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:50.238575 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:50.242288 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:50.243356 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:50.243366 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:50.243379 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:50.243645 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:50.244222 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:50.244228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:50.244230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:50.244922 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:50.247542 (+ 2620us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:50.247744 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 07:07:50.248120 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":209,"threads_started":1}
I20260812 07:07:50.253795 32489 catalog_manager.cc:1347] Generated new cluster ID: 96789dd943ac4f6e8b641b1e5780fd44
I20260812 07:07:50.253904 32489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:50.285068 32489 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:50.287582 32489 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:50.301942 32489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 765204bbe6b64a60a91af8628275fd6d: Generated new TSK 0
I20260812 07:07:50.303130 32489 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:07:50.359390 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34069
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:50.529619 32501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:50.529975 32501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:50.530066 32501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:50.534777 32501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:50.534879 32501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:50.534987 32501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:07:50.541682 32501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34069
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:50.543017 32501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:50.544193 32501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:07:50.555735 32501 server_base.cc:1034] running on GCE node
W20260812 07:07:50.555898 32508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:50.555898 32507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:50.556003 32510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:50.556689 32501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:50.557395 32501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:50.558741 32501 hybrid_clock.cc:648] HybridClock initialized: now 1786518470558694 us; error 64 us; skew 500 ppm
Aug 12 07:07:50 dist-test-slave-csg5 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518470, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:50.562000 32501 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:07:50.563083 32501 webserver.cc:460] Webserver started at http://127.31.156.65:33985/ using document root <none> and password file <none>
I20260812 07:07:50.563705 32501 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:50.563776 32501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:50.564028 32501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:50.566335 32501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "26e7ab4177e540769413c82a41117eba"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-csg5"
server_key: "3c240ec0531e077938948d59bcb9dd96"
server_key_iv: "9bc95d0ab1e5630000e3c4cbaad02bf7"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.567087 32501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "26e7ab4177e540769413c82a41117eba"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-csg5"
server_key: "3c240ec0531e077938948d59bcb9dd96"
server_key_iv: "9bc95d0ab1e5630000e3c4cbaad02bf7"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.571692 32501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:07:50.575042 32518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:50.576926 32501 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:07:50.577052 32501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "26e7ab4177e540769413c82a41117eba"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-csg5"
server_key: "3c240ec0531e077938948d59bcb9dd96"
server_key_iv: "9bc95d0ab1e5630000e3c4cbaad02bf7"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.577251 32501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:50.597353 32501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:50.601313 32501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:50.601548 32501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:50.602424 32501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:50.603487 32501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:50.603540 32501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:50.603582 32501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:50.603634 32501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:50.613296 32501 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:42281
I20260812 07:07:50.613323 32645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:42281 every 8 connection(s)
I20260812 07:07:50.614327 32501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:07:50.620139 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 32501
I20260812 07:07:50.620249 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:07:50.620558 32369 external_mini_cluster.cc:1428] Setting key 160e24ea79342d5312bea7739693f7bc
Aug 12 07:07:50 dist-test-slave-csg5 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518470, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:50.632862 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:50.616526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:51485 (local address 127.31.156.126:34069)
0812 07:07:50.616682 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:50.616686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:50.617570 (+ 884us) server_negotiation.cc:407] Connection header received
0812 07:07:50.619217 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:50.619221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:50.619291 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:50.619383 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:50.620985 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.621769 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.623617 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.623931 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.624049 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:50.627528 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:50.627531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:50.627533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:50.627562 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:50.629673 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:50.630604 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:50.630607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:50.630609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:50.630840 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:50.631440 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:50.631443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:50.631444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:50.631668 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:50.631757 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:50.632585 (+ 828us) server_negotiation.cc:299] Negotiation successful
0812 07:07:50.632721 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:07:50.634066 32648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:50.616881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:34069 (local address 127.31.156.65:51485)
0812 07:07:50.617386 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:50.617437 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:50.618950 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:50.619601 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:50.619608 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:50.620043 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:50.620799 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:50.620819 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.621933 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.621942 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:50.623430 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:50.623448 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.624144 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.624152 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:50.624450 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:50.625357 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:50.625376 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:50.627284 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:50.629851 (+ 2567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:50.629861 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:50.629889 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:50.630383 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:50.630967 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:50.630971 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:50.630973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:50.631215 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:50.631799 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:50.631810 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:50.632248 (+ 438us) client_negotiation.cc:769] Sending connection context
0812 07:07:50.632509 (+ 261us) client_negotiation.cc:240] Negotiation successful
0812 07:07:50.632966 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":124,"threads_started":1}
I20260812 07:07:50.636067 32646 heartbeater.cc:347] Connected to a master server at 127.31.156.126:34069
I20260812 07:07:50.636445 32646 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
I20260812 07:07:50.637074 32646 heartbeater.cc:511] Master 127.31.156.126:34069 requested a full tablet report, sending...
I20260812 07:07:50.639163 32431 ts_manager.cc:194] Registered new tserver with Master: 26e7ab4177e540769413c82a41117eba (127.31.156.65:42281)
I20260812 07:07:50.640690 32431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:51485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:07:50.659875 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34069
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:50.783438 32652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:50.783787 32652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:50.783874 32652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:50.788187 32652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:50.788302 32652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:50.788404 32652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:07:50.794210 32652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34069
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:50.795626 32652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:50.796758 32652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:50.807791 32657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:50.807943 32652 server_base.cc:1034] running on GCE node
W20260812 07:07:50.807806 32659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:50.807770 32661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:50.808552 32652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:50.809383 32652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:50.810599 32652 hybrid_clock.cc:648] HybridClock initialized: now 1786518470810499 us; error 112 us; skew 500 ppm
Aug 12 07:07:50 dist-test-slave-csg5 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518470, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:50.814275 32652 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:07:50.815505 32652 webserver.cc:460] Webserver started at http://127.31.156.66:36567/ using document root <none> and password file <none>
I20260812 07:07:50.816205 32652 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:50.816289 32652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:50.816561 32652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:50.819283 32652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "236295f912c74084a983a1e2d624dffe"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-csg5"
server_key: "7cbf8079ccaa858cceaeaeea50e8e935"
server_key_iv: "a0a5711c2b5078097d34b75ccf660d97"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.820055 32652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "236295f912c74084a983a1e2d624dffe"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-csg5"
server_key: "7cbf8079ccaa858cceaeaeea50e8e935"
server_key_iv: "a0a5711c2b5078097d34b75ccf660d97"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.825204 32652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:07:50.828619 32669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:50.830451 32652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:07:50.830593 32652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "236295f912c74084a983a1e2d624dffe"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-csg5"
server_key: "7cbf8079ccaa858cceaeaeea50e8e935"
server_key_iv: "a0a5711c2b5078097d34b75ccf660d97"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.830816 32652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:50.848737 32652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:50.853211 32652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:50.853463 32652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:50.854249 32652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:50.855360 32652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:50.855444 32652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:50.855525 32652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:50.855574 32652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:50.866133 32652 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:40115
I20260812 07:07:50.866160 330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:40115 every 8 connection(s)
I20260812 07:07:50.867230 32652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:07:50.869051 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 32652
I20260812 07:07:50.869185 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:07:50.869499 32369 external_mini_cluster.cc:1428] Setting key 5695aa53e680afa6e48484c07ac2c31f
Aug 12 07:07:50 dist-test-slave-csg5 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518470, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
I20260812 07:07:50.886615 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:50.869570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:54949 (local address 127.31.156.126:34069)
0812 07:07:50.869734 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:50.869738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:50.870938 (+ 1200us) server_negotiation.cc:407] Connection header received
0812 07:07:50.872543 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:50.872550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:50.872621 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:50.872746 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:50.874597 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.875298 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.877445 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.877693 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.877849 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:50.881522 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:50.881529 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:50.881535 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:50.881578 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:50.883618 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:50.884397 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:50.884400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:50.884402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:50.884503 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:50.884972 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:50.884975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:50.884977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:50.885395 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:50.885527 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:50.886249 (+ 722us) server_negotiation.cc:299] Negotiation successful
0812 07:07:50.886397 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:07:50.887751 333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:50.870091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:34069 (local address 127.31.156.66:54949)
0812 07:07:50.870731 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:50.870806 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:50.872297 (+ 1491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:50.872953 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:50.872962 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:50.873560 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:50.874392 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:50.874417 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.875473 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.875481 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:50.877285 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:50.877303 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.877955 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.877964 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:50.878251 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:50.879100 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:50.879135 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:50.881273 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:50.883756 (+ 2483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:50.883762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:50.883779 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:50.884116 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:50.884656 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:50.884660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:50.884662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:50.884836 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:50.885554 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:50.885564 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:50.885918 (+ 354us) client_negotiation.cc:769] Sending connection context
0812 07:07:50.886127 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 07:07:50.886610 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":131,"threads_started":1}
I20260812 07:07:50.890261 331 heartbeater.cc:347] Connected to a master server at 127.31.156.126:34069
I20260812 07:07:50.890573 331 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:50.891304 331 heartbeater.cc:511] Master 127.31.156.126:34069 requested a full tablet report, sending...
I20260812 07:07:50.892701 32431 ts_manager.cc:194] Registered new tserver with Master: 236295f912c74084a983a1e2d624dffe (127.31.156.66:40115)
I20260812 07:07:50.893409 32431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:54949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:07:50.911917 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34069
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:51.043642 337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:51.043978 337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:51.044030 337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:51.048992 337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:51.049098 337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:51.049171 337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:07:51.055600 337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34069
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:51.056766 337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:51.057814 337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:51.068624 343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:51.068663 345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:51.068598 342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:51.069221 337 server_base.cc:1034] running on GCE node
I20260812 07:07:51.069753 337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:51.070575 337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:51.071775 337 hybrid_clock.cc:648] HybridClock initialized: now 1786518471071762 us; error 30 us; skew 500 ppm
Aug 12 07:07:51 dist-test-slave-csg5 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518471, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:51.075269 337 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:07:51.076368 337 webserver.cc:460] Webserver started at http://127.31.156.67:39599/ using document root <none> and password file <none>
I20260812 07:07:51.077047 337 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:51.077121 337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:51.077400 337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:51.079905 337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "f823ad8d81c8422ca47454ab58ab0786"
format_stamp: "Formatted at 2026-08-12 07:07:51 on dist-test-slave-csg5"
server_key: "5159247fdea1b5666daf5979551191f0"
server_key_iv: "b68d752f9fc2885e70634007767403bf"
server_key_version: "encryptionkey@0"
I20260812 07:07:51.080580 337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "f823ad8d81c8422ca47454ab58ab0786"
format_stamp: "Formatted at 2026-08-12 07:07:51 on dist-test-slave-csg5"
server_key: "5159247fdea1b5666daf5979551191f0"
server_key_iv: "b68d752f9fc2885e70634007767403bf"
server_key_version: "encryptionkey@0"
I20260812 07:07:51.085204 337 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:07:51.088723 352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:51.090473 337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:07:51.090763 337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "f823ad8d81c8422ca47454ab58ab0786"
format_stamp: "Formatted at 2026-08-12 07:07:51 on dist-test-slave-csg5"
server_key: "5159247fdea1b5666daf5979551191f0"
server_key_iv: "b68d752f9fc2885e70634007767403bf"
server_key_version: "encryptionkey@0"
I20260812 07:07:51.090939 337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:51.125319 337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:51.129594 337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:51.129870 337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:51.131476 337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:51.132951 337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:51.133049 337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:51.133122 337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:51.133175 337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:51.144750 337 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:43207
I20260812 07:07:51.144860 477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:43207 every 8 connection(s)
I20260812 07:07:51.145856 337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:07:51.146366 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 337
I20260812 07:07:51.146495 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:07:51.146860 32369 external_mini_cluster.cc:1428] Setting key 7b730e55f48b9f4c478573537f3bbbda
Aug 12 07:07:51 dist-test-slave-csg5 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518471, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:51.162940 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:51.148181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:56035 (local address 127.31.156.126:34069)
0812 07:07:51.148452 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:51.148457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:51.150149 (+ 1692us) server_negotiation.cc:407] Connection header received
0812 07:07:51.151232 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:51.151235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:51.151379 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:51.151458 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:51.153056 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:51.153700 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:51.155307 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:51.155552 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:51.155643 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:51.158978 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:51.158982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:51.158985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:51.159016 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:51.160533 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:51.161191 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:51.161216 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:51.161218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:51.161306 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:51.161703 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:51.161707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:51.161709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:51.161919 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:51.162035 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:51.162590 (+ 555us) server_negotiation.cc:299] Negotiation successful
0812 07:07:51.162762 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:07:51.163934 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:51.148775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:34069 (local address 127.31.156.67:56035)
0812 07:07:51.150003 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:51.150059 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:51.151020 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:51.151786 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:51.151794 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:51.152321 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:51.152877 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:51.152892 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:51.153878 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:51.153881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:51.155168 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:51.155187 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:51.155755 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:51.155758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:51.155942 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:51.156759 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:51.156779 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:51.158746 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:51.160682 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:51.160689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:51.160707 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:51.161067 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:51.161421 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:51.161424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:51.161426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:51.161591 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:51.162034 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:51.162043 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:51.162331 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 07:07:51.162534 (+ 203us) client_negotiation.cc:240] Negotiation successful
0812 07:07:51.162955 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":691,"threads_started":1}
I20260812 07:07:51.165182 478 heartbeater.cc:347] Connected to a master server at 127.31.156.126:34069
I20260812 07:07:51.165525 478 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:51.166145 478 heartbeater.cc:511] Master 127.31.156.126:34069 requested a full tablet report, sending...
I20260812 07:07:51.167483 32431 ts_manager.cc:194] Registered new tserver with Master: f823ad8d81c8422ca47454ab58ab0786 (127.31.156.67:43207)
I20260812 07:07:51.168104 32431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:56035
I20260812 07:07:51.174559 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 07:07:51 dist-test-slave-csg5 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518471, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:07:51 dist-test-slave-csg5 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518471, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:07:51.209003 487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:51.196946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34802 (local address 127.31.156.65:42281)
0812 07:07:51.197298 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:51.197307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:51.197423 (+ 116us) server_negotiation.cc:407] Connection header received
0812 07:07:51.197614 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:51.197621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:51.197831 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:51.197996 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:51.198955 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:51.199719 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:51.200640 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:51.201046 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:51.201169 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:51.204311 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:51.204329 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:51.204350 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:51.204389 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:51.206426 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:51.207113 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:51.207121 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:51.207123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:51.207251 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:51.207654 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:51.207658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:51.207660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:51.208101 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:51.208347 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:51.208600 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 07:07:51.208795 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":96,"threads_started":1}
W20260812 07:07:51.210197 32550 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34802
Aug 12 07:07:51 dist-test-slave-csg5 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:07:51 dist-test-slave-csg5 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:07:51.242806 487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:51.233101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34818 (local address 127.31.156.65:42281)
0812 07:07:51.233272 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:51.233277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:51.233537 (+ 260us) server_negotiation.cc:407] Connection header received
0812 07:07:51.233711 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:51.233714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:51.233770 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:51.233876 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:51.234637 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:51.235260 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:51.236140 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:51.236378 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:51.236493 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:51.239266 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:51.239270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:51.239272 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:51.239303 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:51.241006 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:51.241482 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:51.241486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:51.241488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:51.241581 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:51.241873 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:51.241876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:51.241878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:51.242074 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:51.242223 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:51.242475 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 07:07:51.242605 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:07:51.244079 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 32501
I20260812 07:07:51.254578 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 32652
I20260812 07:07:51.265746 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 337
I20260812 07:07:51.276151 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 32393
2026-08-12T07:07:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2708 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:51 dist-test-slave-csg5 krb5kdc[494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:51 dist-test-slave-csg5 krb5kdc[494](info): set up 2 sockets
Aug 12 07:07:51 dist-test-slave-csg5 krb5kdc[494](info): commencing operation
krb5kdc: starting...
W20260812 07:07:52.509052 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:07:52 dist-test-slave-csg5 krb5kdc[494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:07:52.666968 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45847
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:45847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:52.793764 508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:52.794155 508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:52.794247 508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:52.798542 508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:52.798676 508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:52.798746 508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:52.798808 508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:52.798867 508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:52.804710 508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:45847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:52.806034 508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:52.807179 508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:52.816493 514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:52.816493 513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:52.816781 508 server_base.cc:1034] running on GCE node
W20260812 07:07:52.816969 517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:52.817679 508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:52.819067 508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:52.820354 508 hybrid_clock.cc:648] HybridClock initialized: now 1786518472820318 us; error 54 us; skew 500 ppm
Aug 12 07:07:52 dist-test-slave-csg5 krb5kdc[494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518472, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:52.824020 508 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:07:52.825286 508 webserver.cc:460] Webserver started at http://127.31.156.126:40455/ using document root <none> and password file <none>
I20260812 07:07:52.825937 508 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:52.826005 508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:52.826264 508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:52.828650 508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "fcadc5cd0ff94220857aabd70bdc6651"
format_stamp: "Formatted at 2026-08-12 07:07:52 on dist-test-slave-csg5"
server_key: "69bc9fba99b6aa07b82e3c681e34c8a0"
server_key_iv: "db4ecd7f234aa1116b98e1959ebe1b00"
server_key_version: "encryptionkey@0"
I20260812 07:07:52.829278 508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "fcadc5cd0ff94220857aabd70bdc6651"
format_stamp: "Formatted at 2026-08-12 07:07:52 on dist-test-slave-csg5"
server_key: "69bc9fba99b6aa07b82e3c681e34c8a0"
server_key_iv: "db4ecd7f234aa1116b98e1959ebe1b00"
server_key_version: "encryptionkey@0"
I20260812 07:07:52.834084 508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:07:52.837294 524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:52.838976 508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:07:52.839154 508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "fcadc5cd0ff94220857aabd70bdc6651"
format_stamp: "Formatted at 2026-08-12 07:07:52 on dist-test-slave-csg5"
server_key: "69bc9fba99b6aa07b82e3c681e34c8a0"
server_key_iv: "db4ecd7f234aa1116b98e1959ebe1b00"
server_key_version: "encryptionkey@0"
I20260812 07:07:52.839303 508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:52.851451 508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:52.855706 508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:52.856007 508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:52.863999 508 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:45847
I20260812 07:07:52.864029 586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:45847 every 8 connection(s)
I20260812 07:07:52.865062 508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:52.865732 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 508
I20260812 07:07:52.865872 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:07:52.866148 32369 external_mini_cluster.cc:1428] Setting key 4396b590b39c802d92041642341ee28a
I20260812 07:07:52.869024 587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:52.876070 587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651: Bootstrap starting.
Aug 12 07:07:52 dist-test-slave-csg5 krb5kdc[494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:52.879256 587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:52.880185 587 log.cc:824] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:52.882869 587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651: No bootstrap required, opened a new log
I20260812 07:07:52.883200 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:52.867811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.31.156.126:45847)
0812 07:07:52.868507 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:52.868525 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:52.868564 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:07:52.869344 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:52.869364 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:52.869706 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:52.870064 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:52.871001 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:52.872032 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:52.872941 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:52.873449 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:52.873639 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:52.876989 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:52.876995 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:52.877013 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:52.877047 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:52.879289 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:52.879942 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:52.879957 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:52.879963 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:52.880108 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:52.880532 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:52.880535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:52.880538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:52.881261 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:52.881513 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:52.881866 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 07:07:52.882225 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":173,"threads_started":1}
I20260812 07:07:52.887531 587 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcadc5cd0ff94220857aabd70bdc6651" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45847 } }
I20260812 07:07:52.887833 587 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:52.887928 587 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcadc5cd0ff94220857aabd70bdc6651, State: Initialized, Role: FOLLOWER
I20260812 07:07:52.888438 587 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcadc5cd0ff94220857aabd70bdc6651" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45847 } }
I20260812 07:07:52.888602 587 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:52.888690 587 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:52.888794 587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:52.889910 587 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcadc5cd0ff94220857aabd70bdc6651" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45847 } }
I20260812 07:07:52.890321 587 leader_election.cc:302] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcadc5cd0ff94220857aabd70bdc6651; no voters:
I20260812 07:07:52.890724 587 leader_election.cc:288] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:52.890908 592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:52.891196 592 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [term 1 LEADER]: Becoming Leader. State: Replica: fcadc5cd0ff94220857aabd70bdc6651, State: Running, Role: LEADER
I20260812 07:07:52.891602 592 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcadc5cd0ff94220857aabd70bdc6651" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45847 } }
I20260812 07:07:52.892132 587 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:52.893491 594 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcadc5cd0ff94220857aabd70bdc6651. Latest consensus state: current_term: 1 leader_uuid: "fcadc5cd0ff94220857aabd70bdc6651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcadc5cd0ff94220857aabd70bdc6651" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45847 } } }
I20260812 07:07:52.893637 594 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:52.894064 593 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcadc5cd0ff94220857aabd70bdc6651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcadc5cd0ff94220857aabd70bdc6651" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45847 } } }
I20260812 07:07:52.894142 603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:52.894174 593 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:52.897979 603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:52.904369 603 catalog_manager.cc:1347] Generated new cluster ID: 93475da6d2ae4fe6afb361a5bff77028
I20260812 07:07:52.904469 603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:52.914500 603 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:52.915640 603 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:52.922253 603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcadc5cd0ff94220857aabd70bdc6651: Generated new TSK 0
I20260812 07:07:52.923179 603 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:07:52.963214 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45847
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:53.087746 615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:53.088173 615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:53.088231 615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:53.092845 615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:53.092937 615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:53.093032 615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:07:53.099152 615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:53.100425 615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:53.101547 615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:53.112586 624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:53.112596 620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:53.112658 615 server_base.cc:1034] running on GCE node
W20260812 07:07:53.112591 622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:53.113516 615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:53.114231 615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:53.115463 615 hybrid_clock.cc:648] HybridClock initialized: now 1786518473115422 us; error 60 us; skew 500 ppm
Aug 12 07:07:53 dist-test-slave-csg5 krb5kdc[494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518473, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:53.118937 615 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:07:53.120033 615 webserver.cc:460] Webserver started at http://127.31.156.65:34215/ using document root <none> and password file <none>
I20260812 07:07:53.120684 615 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:53.120805 615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:53.121062 615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:53.123512 615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "cf9f6c0a2c1e4a43b51b2c711379712a"
format_stamp: "Formatted at 2026-08-12 07:07:53 on dist-test-slave-csg5"
server_key: "6dfc656ba1c60b4431a43636c9e4ef2a"
server_key_iv: "597e10cc8c3f49e1257288a11311474b"
server_key_version: "encryptionkey@0"
I20260812 07:07:53.124207 615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "cf9f6c0a2c1e4a43b51b2c711379712a"
format_stamp: "Formatted at 2026-08-12 07:07:53 on dist-test-slave-csg5"
server_key: "6dfc656ba1c60b4431a43636c9e4ef2a"
server_key_iv: "597e10cc8c3f49e1257288a11311474b"
server_key_version: "encryptionkey@0"
I20260812 07:07:53.128918 615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 07:07:53.132104 631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:53.134025 615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:07:53.134227 615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "cf9f6c0a2c1e4a43b51b2c711379712a"
format_stamp: "Formatted at 2026-08-12 07:07:53 on dist-test-slave-csg5"
server_key: "6dfc656ba1c60b4431a43636c9e4ef2a"
server_key_iv: "597e10cc8c3f49e1257288a11311474b"
server_key_version: "encryptionkey@0"
I20260812 07:07:53.134383 615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:53.159806 615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:53.164213 615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:53.164508 615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:53.165328 615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:53.166347 615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:53.166431 615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:53.166513 615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:53.166566 615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:53.177379 615 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:44713
I20260812 07:07:53.177481 762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:44713 every 8 connection(s)
I20260812 07:07:53.178436 615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:07:53.182526 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 615
I20260812 07:07:53.182695 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:07:53.182984 32369 external_mini_cluster.cc:1428] Setting key 47d64f418bec216e1b8e1c1ce3cec500
Aug 12 07:07:53 dist-test-slave-csg5 krb5kdc[494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518473, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:07:53.197589 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.180729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:43875 (local address 127.31.156.126:45847)
0812 07:07:53.180898 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:53.180903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:53.181993 (+ 1090us) server_negotiation.cc:407] Connection header received
0812 07:07:53.183763 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:53.183767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:53.183834 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:53.183904 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:53.185680 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.186552 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.188220 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.188561 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.188638 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.192580 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:53.192588 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:53.192591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:53.192635 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:53.194749 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:53.195484 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:53.195488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:53.195490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:53.195598 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:53.196077 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:53.196081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:53.196083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:53.196334 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:53.196464 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:53.197154 (+ 690us) server_negotiation.cc:299] Negotiation successful
0812 07:07:53.197378 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:07:53.198635 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.181189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45847 (local address 127.31.156.65:43875)
0812 07:07:53.181819 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:53.181869 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:53.183375 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:53.184252 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:53.184262 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:53.184922 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:53.185486 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.185502 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.186811 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.186814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.187999 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.188013 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.188876 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.188879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.189060 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.189977 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:53.189996 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:53.192249 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:53.194950 (+ 2701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:53.194958 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:53.194981 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:53.195341 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:53.195714 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:53.195717 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:53.195719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:53.195949 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:53.196464 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:53.196474 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:53.196831 (+ 357us) client_negotiation.cc:769] Sending connection context
0812 07:07:53.197084 (+ 253us) client_negotiation.cc:240] Negotiation successful
0812 07:07:53.197522 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":127,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
I20260812 07:07:53.200322 763 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45847
I20260812 07:07:53.200632 763 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:53.201249 763 heartbeater.cc:511] Master 127.31.156.126:45847 requested a full tablet report, sending...
I20260812 07:07:53.203238 542 ts_manager.cc:194] Registered new tserver with Master: cf9f6c0a2c1e4a43b51b2c711379712a (127.31.156.65:44713)
I20260812 07:07:53.205076 542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:43875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:07:53.223922 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45847
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:53.352406 768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:53.352802 768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:53.352895 768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:53.357316 768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:53.357440 768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:53.357546 768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:07:53.363421 768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:53.364687 768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:53.365833 768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:53.376695 773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:53.376827 768 server_base.cc:1034] running on GCE node
W20260812 07:07:53.376736 776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:53.376704 774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:53.377457 768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:53.378257 768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:53.379571 768 hybrid_clock.cc:648] HybridClock initialized: now 1786518473379540 us; error 44 us; skew 500 ppm
Aug 12 07:07:53 dist-test-slave-csg5 krb5kdc[494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518473, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:53.388011 768 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:07:53.389290 768 webserver.cc:460] Webserver started at http://127.31.156.66:33511/ using document root <none> and password file <none>
I20260812 07:07:53.389973 768 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:53.390094 768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:53.390506 768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:53.393371 768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "4c5dc91421854753a4c9c421ff912ab0"
format_stamp: "Formatted at 2026-08-12 07:07:53 on dist-test-slave-csg5"
server_key: "f87e3cca92a86c4305f4d3af3899d826"
server_key_iv: "2f08db2407154566fb299aee2eb808fa"
server_key_version: "encryptionkey@0"
I20260812 07:07:53.394147 768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "4c5dc91421854753a4c9c421ff912ab0"
format_stamp: "Formatted at 2026-08-12 07:07:53 on dist-test-slave-csg5"
server_key: "f87e3cca92a86c4305f4d3af3899d826"
server_key_iv: "2f08db2407154566fb299aee2eb808fa"
server_key_version: "encryptionkey@0"
I20260812 07:07:53.399191 768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 07:07:53.402475 786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:53.404181 768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:07:53.404366 768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "4c5dc91421854753a4c9c421ff912ab0"
format_stamp: "Formatted at 2026-08-12 07:07:53 on dist-test-slave-csg5"
server_key: "f87e3cca92a86c4305f4d3af3899d826"
server_key_iv: "2f08db2407154566fb299aee2eb808fa"
server_key_version: "encryptionkey@0"
I20260812 07:07:53.404518 768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:53.423729 768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:53.428542 768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:53.428805 768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:53.429509 768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:53.430526 768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:53.430608 768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:53.430706 768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:53.430754 768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260812 07:07:53.443817 768 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:37957
I20260812 07:07:53.443918 912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:37957 every 8 connection(s)
I20260812 07:07:53.445142 768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:07:53.454366 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 768
I20260812 07:07:53.454483 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:07:53.454869 32369 external_mini_cluster.cc:1428] Setting key d25416e0b88246692fdef98512b3f20c
Aug 12 07:07:53 dist-test-slave-csg5 krb5kdc[494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518473, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:53.464893 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.447780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:57895 (local address 127.31.156.126:45847)
0812 07:07:53.447960 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:53.447965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:53.449468 (+ 1503us) server_negotiation.cc:407] Connection header received
0812 07:07:53.451588 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:53.451592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:53.451654 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:53.451757 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:53.453315 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.453889 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.456015 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.456340 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.456470 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.459854 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:53.459859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:53.459861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:53.459892 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:53.461722 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:53.462567 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:53.462571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:53.462572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:53.462756 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:53.463292 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:53.463295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:53.463297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:53.463560 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:53.463660 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:53.464552 (+ 892us) server_negotiation.cc:299] Negotiation successful
0812 07:07:53.464698 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:07:53.466138 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.448277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45847 (local address 127.31.156.66:57895)
0812 07:07:53.449168 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:53.449262 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:53.451194 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:53.452011 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:53.452019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:53.452471 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:53.453040 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.453055 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.454052 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.454055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.455856 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.455871 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.456580 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.456588 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.456867 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.457599 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:53.457618 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:53.459563 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:53.461892 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:53.461903 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:53.461926 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:53.462416 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:53.462865 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:53.462869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:53.462871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:53.463129 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:53.463692 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:53.463704 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:53.464147 (+ 443us) client_negotiation.cc:769] Sending connection context
0812 07:07:53.464416 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 07:07:53.464860 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":147,"threads_started":1}
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:07:53.467792 913 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45847
I20260812 07:07:53.468086 913 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:53.468883 913 heartbeater.cc:511] Master 127.31.156.126:45847 requested a full tablet report, sending...
I20260812 07:07:53.470410 542 ts_manager.cc:194] Registered new tserver with Master: 4c5dc91421854753a4c9c421ff912ab0 (127.31.156.66:37957)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
I20260812 07:07:53.471130 542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:57895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:07:53.496043 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45847
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:53.626371 918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:53.626763 918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:53.626863 918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:53.631210 918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:53.631330 918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:53.631428 918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:07:53.637328 918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:53.638700 918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:53.640195 918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:53.651893 927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:53.651893 929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:53.651916 926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:53.652498 918 server_base.cc:1034] running on GCE node
I20260812 07:07:53.653067 918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:53.654075 918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:53.655293 918 hybrid_clock.cc:648] HybridClock initialized: now 1786518473655270 us; error 36 us; skew 500 ppm
Aug 12 07:07:53 dist-test-slave-csg5 krb5kdc[494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518473, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:53.659026 918 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:07:53.660151 918 webserver.cc:460] Webserver started at http://127.31.156.67:41871/ using document root <none> and password file <none>
I20260812 07:07:53.660845 918 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:53.660916 918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:53.661177 918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:53.663693 918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "818a1befccbb403082257971dc2694c5"
format_stamp: "Formatted at 2026-08-12 07:07:53 on dist-test-slave-csg5"
server_key: "a4a4e2178c7c866493131c21fd6d59c8"
server_key_iv: "7a754d9ad88e5478f8889a338757b263"
server_key_version: "encryptionkey@0"
I20260812 07:07:53.664610 918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "818a1befccbb403082257971dc2694c5"
format_stamp: "Formatted at 2026-08-12 07:07:53 on dist-test-slave-csg5"
server_key: "a4a4e2178c7c866493131c21fd6d59c8"
server_key_iv: "7a754d9ad88e5478f8889a338757b263"
server_key_version: "encryptionkey@0"
I20260812 07:07:53.669199 918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:07:53.672452 936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:53.674206 918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:07:53.674504 918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "818a1befccbb403082257971dc2694c5"
format_stamp: "Formatted at 2026-08-12 07:07:53 on dist-test-slave-csg5"
server_key: "a4a4e2178c7c866493131c21fd6d59c8"
server_key_iv: "7a754d9ad88e5478f8889a338757b263"
server_key_version: "encryptionkey@0"
I20260812 07:07:53.674759 918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:53.687726 918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:53.691704 918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:53.691977 918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:53.692678 918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:53.693754 918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:53.693827 918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:53.693899 918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:53.693948 918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:53.704752 918 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:41899
I20260812 07:07:53.704905 1063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:41899 every 8 connection(s)
I20260812 07:07:53.705793 918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:07:53.715703 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 918
I20260812 07:07:53.715883 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:07:53.716257 32369 external_mini_cluster.cc:1428] Setting key 8e8ec83da656ac4eb939360bd74773e2
Aug 12 07:07:53 dist-test-slave-csg5 krb5kdc[494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518473, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:53.722479 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.708072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:37741 (local address 127.31.156.126:45847)
0812 07:07:53.708233 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:53.708239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:53.709190 (+ 951us) server_negotiation.cc:407] Connection header received
0812 07:07:53.710887 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:53.710891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:53.710962 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:53.711083 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:53.712444 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.713044 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.714389 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.714648 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.714841 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.718130 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:53.718134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:53.718136 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:53.718169 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:53.719982 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:53.720649 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:53.720653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:53.720654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:53.720748 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:53.721178 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:53.721184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:53.721186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:53.721414 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:53.721540 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:53.722152 (+ 612us) server_negotiation.cc:299] Negotiation successful
0812 07:07:53.722296 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:07:53.723495 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.708379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45847 (local address 127.31.156.67:37741)
0812 07:07:53.708996 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:53.709045 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:53.710606 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:53.711283 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:53.711291 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:53.711699 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:53.712269 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.712284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.713195 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.713201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.714237 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.714248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.714896 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.714901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.715106 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.715919 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:53.715947 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:53.717916 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:53.720132 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:53.720140 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:53.720159 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:53.720492 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:53.720882 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:53.720885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:53.720888 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:53.721065 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:53.721539 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:53.721549 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:53.721858 (+ 309us) client_negotiation.cc:769] Sending connection context
0812 07:07:53.722098 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 07:07:53.722502 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":195,"threads_started":1}
I20260812 07:07:53.725100 1064 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45847
I20260812 07:07:53.725440 1064 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:53.725980 1064 heartbeater.cc:511] Master 127.31.156.126:45847 requested a full tablet report, sending...
I20260812 07:07:53.727222 542 ts_manager.cc:194] Registered new tserver with Master: 818a1befccbb403082257971dc2694c5 (127.31.156.67:41899)
I20260812 07:07:53.727852 542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:37741
I20260812 07:07:53.731240 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:07:53.743963 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.735005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47390 (local address 127.31.156.126:45847)
0812 07:07:53.735139 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:53.735143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:53.735475 (+ 332us) server_negotiation.cc:407] Connection header received
0812 07:07:53.735620 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:53.735623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:53.735670 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:53.735749 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:53.736569 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.737121 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.738025 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.738247 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.738338 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.740257 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:53.740260 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:53.740262 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:53.740290 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:53.741906 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:53.742410 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:53.742413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:53.742414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:53.742510 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:53.742963 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:53.742966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:53.742967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:53.743324 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:53.743494 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:53.743716 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 07:07:53.743828 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:07:53.750012 542 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:07:53.753551 542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:07:53.770068 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.763575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:37957 (local address 127.0.0.1:48000)
0812 07:07:53.764890 (+ 1315us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:53.764937 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:53.765037 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:53.765498 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:53.765501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:53.765519 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:53.765652 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.765659 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.767219 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.767222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.768609 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.768617 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.769454 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.769458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.769582 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.769622 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 07:07:53.769761 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 07:07:53.769902 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1238,"mutex_wait_us":102,"thread_start_us":113,"threads_started":1}
I20260812 07:07:53.770072 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.764012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48000 (local address 127.31.156.66:37957)
0812 07:07:53.764696 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:53.764728 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:53.765046 (+ 318us) server_negotiation.cc:407] Connection header received
0812 07:07:53.765151 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:53.765156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:53.765339 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:53.765490 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:53.765773 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.766879 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.768753 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.769324 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.769452 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.769615 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:53.769794 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 07:07:53.769897 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":118,"threads_started":1}
I20260812 07:07:53.770537 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.763316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:41899 (local address 127.0.0.1:45556)
0812 07:07:53.764145 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:53.764158 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:53.764288 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:53.765861 (+ 1573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:53.765863 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:53.765878 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:53.765957 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.765961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.767118 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.767122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.768606 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.768617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.770076 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.770081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.770214 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.770236 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 07:07:53.770309 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 07:07:53.770376 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":102,"spinlock_wait_cycles":3200,"thread_start_us":114,"threads_started":1}
I20260812 07:07:53.770964 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.763365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53440 (local address 127.31.156.65:44713)
0812 07:07:53.763756 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:53.763763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:53.764165 (+ 402us) server_negotiation.cc:407] Connection header received
0812 07:07:53.764479 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:53.764484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:53.764686 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:53.764814 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:53.765460 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.766734 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.768533 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.769445 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.769574 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.769823 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:53.770615 (+ 792us) server_negotiation.cc:299] Negotiation successful
0812 07:07:53.770783 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":99,"threads_started":1}
I20260812 07:07:53.771226 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.764012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45556 (local address 127.31.156.67:41899)
0812 07:07:53.765262 (+ 1250us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:53.765270 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:53.765288 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:07:53.765348 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:53.765351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:53.765533 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:53.765683 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:53.766085 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.766978 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.768751 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.769736 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.769869 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.770014 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:53.770934 (+ 920us) server_negotiation.cc:299] Negotiation successful
0812 07:07:53.771058 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1158,"thread_start_us":168,"threads_started":1}
I20260812 07:07:53.770485 1077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.763182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:44713 (local address 127.0.0.1:53440)
0812 07:07:53.764020 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:53.764076 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:53.764288 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:53.765122 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:53.765129 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:53.765155 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:53.765312 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.765319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.766857 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.766860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.768358 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.768370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.769903 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.769907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.770039 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.770068 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:07:53.770146 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:07:53.770233 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":131,"spinlock_wait_cycles":7936,"thread_start_us":102,"threads_started":1}
I20260812 07:07:53.774680 686 tablet_service.cc:1467] Processing CreateTablet for tablet 7ccf450d355543d590c1fab4b1c01905 (DEFAULT_TABLE table=test-table [id=96f2e38d09c54f5fa4a50b2ee1be1f8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:53.774518 841 tablet_service.cc:1467] Processing CreateTablet for tablet 7ccf450d355543d590c1fab4b1c01905 (DEFAULT_TABLE table=test-table [id=96f2e38d09c54f5fa4a50b2ee1be1f8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:53.775827 686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ccf450d355543d590c1fab4b1c01905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:53.775827 841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ccf450d355543d590c1fab4b1c01905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:53.776656 990 tablet_service.cc:1467] Processing CreateTablet for tablet 7ccf450d355543d590c1fab4b1c01905 (DEFAULT_TABLE table=test-table [id=96f2e38d09c54f5fa4a50b2ee1be1f8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:53.777621 990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ccf450d355543d590c1fab4b1c01905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:53.783941 1084 tablet_bootstrap.cc:492] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a: Bootstrap starting.
I20260812 07:07:53.785260 1085 tablet_bootstrap.cc:492] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0: Bootstrap starting.
I20260812 07:07:53.785351 1086 tablet_bootstrap.cc:492] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5: Bootstrap starting.
I20260812 07:07:53.787050 1084 tablet_bootstrap.cc:654] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:53.787497 1086 tablet_bootstrap.cc:654] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:53.787981 1085 tablet_bootstrap.cc:654] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:53.788133 1084 log.cc:824] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:53.788806 1086 log.cc:824] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:53.788823 1085 log.cc:824] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:53.790738 1084 tablet_bootstrap.cc:492] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a: No bootstrap required, opened a new log
I20260812 07:07:53.790738 1086 tablet_bootstrap.cc:492] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5: No bootstrap required, opened a new log
I20260812 07:07:53.790964 1084 ts_tablet_manager.cc:1397] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:07:53.790964 1086 ts_tablet_manager.cc:1397] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 07:07:53.792062 1085 tablet_bootstrap.cc:492] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0: No bootstrap required, opened a new log
I20260812 07:07:53.792323 1085 ts_tablet_manager.cc:1397] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 07:07:53.795564 1086 raft_consensus.cc:348] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44713 } } peers { permanent_uuid: "818a1befccbb403082257971dc2694c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41899 } } peers { permanent_uuid: "4c5dc91421854753a4c9c421ff912ab0" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 37957 } }
I20260812 07:07:53.795662 1084 raft_consensus.cc:348] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44713 } } peers { permanent_uuid: "818a1befccbb403082257971dc2694c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41899 } } peers { permanent_uuid: "4c5dc91421854753a4c9c421ff912ab0" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 37957 } }
I20260812 07:07:53.795913 1086 raft_consensus.cc:374] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:53.795913 1084 raft_consensus.cc:374] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:53.795979 1084 raft_consensus.cc:729] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9f6c0a2c1e4a43b51b2c711379712a, State: Initialized, Role: FOLLOWER
I20260812 07:07:53.795989 1086 raft_consensus.cc:729] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 818a1befccbb403082257971dc2694c5, State: Initialized, Role: FOLLOWER
I20260812 07:07:53.795840 1085 raft_consensus.cc:348] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44713 } } peers { permanent_uuid: "818a1befccbb403082257971dc2694c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41899 } } peers { permanent_uuid: "4c5dc91421854753a4c9c421ff912ab0" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 37957 } }
I20260812 07:07:53.796152 1085 raft_consensus.cc:374] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:53.796257 1085 raft_consensus.cc:729] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5dc91421854753a4c9c421ff912ab0, State: Initialized, Role: FOLLOWER
I20260812 07:07:53.796564 1084 consensus_queue.cc:260] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44713 } } peers { permanent_uuid: "818a1befccbb403082257971dc2694c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41899 } } peers { permanent_uuid: "4c5dc91421854753a4c9c421ff912ab0" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 37957 } }
I20260812 07:07:53.796564 1086 consensus_queue.cc:260] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44713 } } peers { permanent_uuid: "818a1befccbb403082257971dc2694c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41899 } } peers { permanent_uuid: "4c5dc91421854753a4c9c421ff912ab0" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 37957 } }
I20260812 07:07:53.796747 1085 consensus_queue.cc:260] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44713 } } peers { permanent_uuid: "818a1befccbb403082257971dc2694c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41899 } } peers { permanent_uuid: "4c5dc91421854753a4c9c421ff912ab0" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 37957 } }
I20260812 07:07:53.797669 1064 heartbeater.cc:503] Master 127.31.156.126:45847 was elected leader, sending a full tablet report...
I20260812 07:07:53.797776 1084 ts_tablet_manager.cc:1428] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:07:53.797776 1086 ts_tablet_manager.cc:1428] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:07:53.797785 1085 ts_tablet_manager.cc:1428] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 07:07:53.798151 913 heartbeater.cc:503] Master 127.31.156.126:45847 was elected leader, sending a full tablet report...
I20260812 07:07:53.799144 763 heartbeater.cc:503] Master 127.31.156.126:45847 was elected leader, sending a full tablet report...
W20260812 07:07:53.930157 764 tablet.cc:2369] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:07:53.946993 914 tablet.cc:2369] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:07:53.957075 1065 tablet.cc:2369] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:07:53.972631 1092 raft_consensus.cc:482] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:53.972913 1092 raft_consensus.cc:504] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44713 } } peers { permanent_uuid: "818a1befccbb403082257971dc2694c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41899 } } peers { permanent_uuid: "4c5dc91421854753a4c9c421ff912ab0" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 37957 } }
I20260812 07:07:53.974227 1092 leader_election.cc:288] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf9f6c0a2c1e4a43b51b2c711379712a (127.31.156.65:44713), 818a1befccbb403082257971dc2694c5 (127.31.156.67:41899)
I20260812 07:07:53.978366 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.974721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:54277 (local address 127.31.156.67:41899)
0812 07:07:53.974871 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:53.974875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:53.975131 (+ 256us) server_negotiation.cc:407] Connection header received
0812 07:07:53.975305 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:53.975308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:53.975348 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:53.975452 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:53.975743 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.976237 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.977323 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.977733 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.977873 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.977930 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:53.978124 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 07:07:53.978199 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:07:53.978389 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.974681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:42377 (local address 127.31.156.65:44713)
0812 07:07:53.974841 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:53.974845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:53.975006 (+ 161us) server_negotiation.cc:407] Connection header received
0812 07:07:53.975186 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:53.975189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:53.975232 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:53.975317 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:53.975598 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.976114 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.977297 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.977740 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.977903 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.977961 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:53.978175 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 07:07:53.978254 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:07:53.978425 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.974516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:44713 (local address 127.31.156.66:42377)
0812 07:07:53.974906 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:53.974918 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:53.975071 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:53.975347 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:53.975350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:53.975366 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:53.975492 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.975498 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.976229 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.976234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.977167 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.977177 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.977903 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.977907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.978046 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.978065 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 07:07:53.978146 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 07:07:53.978256 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":107,"thread_start_us":117,"threads_started":1}
I20260812 07:07:53.978432 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.974528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:41899 (local address 127.31.156.66:54277)
0812 07:07:53.975037 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:53.975049 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:53.975178 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:53.975465 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:53.975468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:53.975485 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:53.975627 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.975632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.976360 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.976365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.977182 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:53.977191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.977873 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.977876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:53.977976 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.978009 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:07:53.978109 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 07:07:53.978208 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260812 07:07:53.979238 1016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ccf450d355543d590c1fab4b1c01905" candidate_uuid: "4c5dc91421854753a4c9c421ff912ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "818a1befccbb403082257971dc2694c5" is_pre_election: true
I20260812 07:07:53.979240 709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ccf450d355543d590c1fab4b1c01905" candidate_uuid: "4c5dc91421854753a4c9c421ff912ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a" is_pre_election: true
I20260812 07:07:53.979576 1016 raft_consensus.cc:2454] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c5dc91421854753a4c9c421ff912ab0 in term 0.
I20260812 07:07:53.979581 709 raft_consensus.cc:2454] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c5dc91421854753a4c9c421ff912ab0 in term 0.
I20260812 07:07:53.980111 789 leader_election.cc:302] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4c5dc91421854753a4c9c421ff912ab0, 818a1befccbb403082257971dc2694c5, cf9f6c0a2c1e4a43b51b2c711379712a; no voters:
I20260812 07:07:53.980432 1092 raft_consensus.cc:2781] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:07:53.980525 1092 raft_consensus.cc:482] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:07:53.980557 1092 raft_consensus.cc:3037] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:53.981678 1092 raft_consensus.cc:504] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44713 } } peers { permanent_uuid: "818a1befccbb403082257971dc2694c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41899 } } peers { permanent_uuid: "4c5dc91421854753a4c9c421ff912ab0" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 37957 } }
I20260812 07:07:53.982115 1092 leader_election.cc:288] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [CANDIDATE]: Term 1 election: Requested vote from peers cf9f6c0a2c1e4a43b51b2c711379712a (127.31.156.65:44713), 818a1befccbb403082257971dc2694c5 (127.31.156.67:41899)
I20260812 07:07:53.982636 709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ccf450d355543d590c1fab4b1c01905" candidate_uuid: "4c5dc91421854753a4c9c421ff912ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a"
I20260812 07:07:53.982828 1016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ccf450d355543d590c1fab4b1c01905" candidate_uuid: "4c5dc91421854753a4c9c421ff912ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "818a1befccbb403082257971dc2694c5"
I20260812 07:07:53.982851 709 raft_consensus.cc:3037] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:53.983004 1016 raft_consensus.cc:3037] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:53.984117 1016 raft_consensus.cc:2454] T 7ccf450d355543d590c1fab4b1c01905 P 818a1befccbb403082257971dc2694c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5dc91421854753a4c9c421ff912ab0 in term 1.
I20260812 07:07:53.984112 709 raft_consensus.cc:2454] T 7ccf450d355543d590c1fab4b1c01905 P cf9f6c0a2c1e4a43b51b2c711379712a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5dc91421854753a4c9c421ff912ab0 in term 1.
I20260812 07:07:53.984472 789 leader_election.cc:302] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c5dc91421854753a4c9c421ff912ab0, 818a1befccbb403082257971dc2694c5; no voters:
I20260812 07:07:53.984673 1092 raft_consensus.cc:2781] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:53.984963 1092 raft_consensus.cc:686] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c5dc91421854753a4c9c421ff912ab0, State: Running, Role: LEADER
I20260812 07:07:53.985338 1092 consensus_queue.cc:237] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44713 } } peers { permanent_uuid: "818a1befccbb403082257971dc2694c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41899 } } peers { permanent_uuid: "4c5dc91421854753a4c9c421ff912ab0" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 37957 } }
I20260812 07:07:53.989069 535 catalog_manager.cc:5495] T 7ccf450d355543d590c1fab4b1c01905 P 4c5dc91421854753a4c9c421ff912ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c5dc91421854753a4c9c421ff912ab0 (127.31.156.66). New cstate: current_term: 1 leader_uuid: "4c5dc91421854753a4c9c421ff912ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9f6c0a2c1e4a43b51b2c711379712a" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "818a1befccbb403082257971dc2694c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5dc91421854753a4c9c421ff912ab0" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 37957 } health_report { overall_health: HEALTHY } } }
Aug 12 07:07:54 dist-test-slave-csg5 krb5kdc[494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:07:54.012931 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:54.003064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53448 (local address 127.31.156.65:44713)
0812 07:07:54.003344 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:54.003348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:54.003364 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:07:54.003412 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:54.003414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:54.003455 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:54.003526 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:54.004244 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:54.004776 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:54.005754 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:54.006016 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:54.006115 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:54.009281 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:54.009314 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:54.009340 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:54.009382 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:54.011151 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:54.011617 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:54.011622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:54.011623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:54.011716 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:54.011968 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:54.011970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:54.011971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:54.012234 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:54.012461 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:54.012587 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 07:07:54.012701 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
Aug 12 07:07:54 dist-test-slave-csg5 krb5kdc[494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518474, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:07:54 dist-test-slave-csg5 krb5kdc[494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518474, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:07:54.046270 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:54.036039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53458 (local address 127.31.156.65:44713)
0812 07:07:54.036240 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:54.036244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:54.036265 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:07:54.036459 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:54.036462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:54.036514 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:54.036597 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:54.037376 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:54.037951 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:54.039153 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:54.039425 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:54.039529 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:54.042557 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:54.042561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:54.042564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:54.042598 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:54.044415 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:54.044973 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:54.044976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:54.044977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:54.045058 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:54.045397 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:54.045399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:54.045400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:54.045577 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:54.045683 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:54.046013 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 07:07:54.046135 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 07:07:54 dist-test-slave-csg5 krb5kdc[494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:07:54 dist-test-slave-csg5 krb5kdc[494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:07:54.078800 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:54.068514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.31.156.65:44713)
0812 07:07:54.068761 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:54.068767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:54.068940 (+ 173us) server_negotiation.cc:407] Connection header received
0812 07:07:54.069141 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:54.069144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:54.069192 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:54.069372 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:54.070192 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:54.070783 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:54.071689 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:54.072045 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:54.072175 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:54.075093 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:54.075097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:54.075100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:54.075139 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:54.076781 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:54.077354 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:54.077357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:54.077358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:54.077445 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:54.077772 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:54.077774 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:54.077775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:54.077950 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:54.078056 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:54.078452 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 07:07:54.078591 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
W20260812 07:07:54.080724 660 tablet_service.cc:2908] Rejecting scan request for tablet 7ccf450d355543d590c1fab4b1c01905: Uninitialized: clean time has not yet been initialized
I20260812 07:07:54.083145 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 615
I20260812 07:07:54.093797 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 768
I20260812 07:07:54.103746 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 918
I20260812 07:07:54.114053 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 508
2026-08-12T07:07:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2838 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:54 dist-test-slave-csg5 krb5kdc[1112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:54 dist-test-slave-csg5 krb5kdc[1112](info): set up 2 sockets
Aug 12 07:07:54 dist-test-slave-csg5 krb5kdc[1112](info): commencing operation
krb5kdc: starting...
W20260812 07:07:55.309962 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:07:55 dist-test-slave-csg5 krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:07:55.484122 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:44441
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:44441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:55.619843 1128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:55.620245 1128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:55.620374 1128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:55.625021 1128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:55.625141 1128 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 07:07:55.625176 1128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:55.625248 1128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:55.625305 1128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:55.625344 1128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:55.631099 1128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:44441
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:44441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:55.632385 1128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:55.633514 1128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:55.643045 1137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:55.643038 1134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:55.643168 1128 server_base.cc:1034] running on GCE node
W20260812 07:07:55.643042 1133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:55.644104 1128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:55.645399 1128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:55.646622 1128 hybrid_clock.cc:648] HybridClock initialized: now 1786518475646607 us; error 30 us; skew 500 ppm
Aug 12 07:07:55 dist-test-slave-csg5 krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518475, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:55.650051 1128 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:07:55.651253 1128 webserver.cc:460] Webserver started at http://127.31.156.126:34655/ using document root <none> and password file <none>
I20260812 07:07:55.651907 1128 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:55.652009 1128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:55.652290 1128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:55.654556 1128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "4f2d6092ae364aeea6ad6b4b0205a7ae"
format_stamp: "Formatted at 2026-08-12 07:07:55 on dist-test-slave-csg5"
server_key: "fae138a984fc782c00670e75c975f88a"
server_key_iv: "ef98c103989104922972818a0451030d"
server_key_version: "encryptionkey@0"
I20260812 07:07:55.655228 1128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "4f2d6092ae364aeea6ad6b4b0205a7ae"
format_stamp: "Formatted at 2026-08-12 07:07:55 on dist-test-slave-csg5"
server_key: "fae138a984fc782c00670e75c975f88a"
server_key_iv: "ef98c103989104922972818a0451030d"
server_key_version: "encryptionkey@0"
I20260812 07:07:55.659535 1128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:07:55.662324 1144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:55.664044 1128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 07:07:55.664408 1128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "4f2d6092ae364aeea6ad6b4b0205a7ae"
format_stamp: "Formatted at 2026-08-12 07:07:55 on dist-test-slave-csg5"
server_key: "fae138a984fc782c00670e75c975f88a"
server_key_iv: "ef98c103989104922972818a0451030d"
server_key_version: "encryptionkey@0"
I20260812 07:07:55.664580 1128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:55.684795 1128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:55.689344 1128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:55.689615 1128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:55.699486 1128 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:44441
I20260812 07:07:55.699556 1215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:44441 every 8 connection(s)
I20260812 07:07:55.700626 1128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:55.703923 1216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:55.703925 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 1128
I20260812 07:07:55.704119 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:07:55.704527 32369 external_mini_cluster.cc:1428] Setting key d0cb1283aed652062a4d245fe35fd2a0
I20260812 07:07:55.712545 1216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae: Bootstrap starting.
I20260812 07:07:55.715670 1216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae: Neither blocks nor log segments found. Creating new log.
Aug 12 07:07:55 dist-test-slave-csg5 krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:55.716591 1216 log.cc:824] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:55.719285 1216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae: No bootstrap required, opened a new log
I20260812 07:07:55.722646 1216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2d6092ae364aeea6ad6b4b0205a7ae" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44441 } }
I20260812 07:07:55.723034 1216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:55.723122 1216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f2d6092ae364aeea6ad6b4b0205a7ae, State: Initialized, Role: FOLLOWER
I20260812 07:07:55.723671 1216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2d6092ae364aeea6ad6b4b0205a7ae" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44441 } }
I20260812 07:07:55.723747 1220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:55.706247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38154 (local address 127.31.156.126:44441)
0812 07:07:55.706819 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:55.706841 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:55.706893 (+ 52us) server_negotiation.cc:407] Connection header received
0812 07:07:55.708135 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:55.708165 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:55.708672 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:55.709115 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:55.710138 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:55.711725 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:55.712954 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:55.713661 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:55.713850 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:55.716847 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:55.716853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:55.716870 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:55.716906 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:55.719749 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:55.720379 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:55.720385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:55.720390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:55.720528 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:55.720922 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:55.720926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:55.720928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:55.721400 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:55.721648 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:55.721985 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 07:07:55.722335 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":123,"threads_started":1}
I20260812 07:07:55.723862 1216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:55.723948 1216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:55.724084 1216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:55.725458 1216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2d6092ae364aeea6ad6b4b0205a7ae" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44441 } }
I20260812 07:07:55.725848 1216 leader_election.cc:302] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f2d6092ae364aeea6ad6b4b0205a7ae; no voters:
I20260812 07:07:55.726223 1216 leader_election.cc:288] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:55.726365 1222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:55.726619 1222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [term 1 LEADER]: Becoming Leader. State: Replica: 4f2d6092ae364aeea6ad6b4b0205a7ae, State: Running, Role: LEADER
I20260812 07:07:55.727103 1222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2d6092ae364aeea6ad6b4b0205a7ae" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44441 } }
I20260812 07:07:55.727715 1216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:55.729423 1225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f2d6092ae364aeea6ad6b4b0205a7ae. Latest consensus state: current_term: 1 leader_uuid: "4f2d6092ae364aeea6ad6b4b0205a7ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2d6092ae364aeea6ad6b4b0205a7ae" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44441 } } }
I20260812 07:07:55.729642 1225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:55.729400 1223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f2d6092ae364aeea6ad6b4b0205a7ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2d6092ae364aeea6ad6b4b0205a7ae" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44441 } } }
I20260812 07:07:55.729887 1223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:55.730191 1239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:55.733860 1239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:55.740091 1239 catalog_manager.cc:1347] Generated new cluster ID: 631b1791a1b843d89cb22d927e887797
I20260812 07:07:55.740224 1239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:55.757171 1239 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:55.758334 1239 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:55.762706 1239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae: Generated new TSK 0
I20260812 07:07:55.763445 1239 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:07:55.806963 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:44441
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:55.935911 1247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:55.936251 1247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:55.936300 1247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:55.941018 1247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:55.941131 1247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:55.941179 1247 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:07:55.941216 1247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:07:55.948028 1247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.156.126:44441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:55.949553 1247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:55.950784 1247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:55.962218 1258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:55.962265 1254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:55.962221 1253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:55.962631 1247 server_base.cc:1034] running on GCE node
I20260812 07:07:55.963176 1247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:55.963965 1247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:55.965229 1247 hybrid_clock.cc:648] HybridClock initialized: now 1786518475965193 us; error 51 us; skew 500 ppm
Aug 12 07:07:55 dist-test-slave-csg5 krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518475, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:55.969529 1247 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:07:55.971024 1247 webserver.cc:460] Webserver started at http://127.31.156.65:34745/ using document root <none> and password file <none>
I20260812 07:07:55.971714 1247 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:55.971839 1247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:55.972127 1247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:55.974782 1247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "bda99d14cdea44a493b660c86ab3282e"
format_stamp: "Formatted at 2026-08-12 07:07:55 on dist-test-slave-csg5"
server_key: "5ab1685de7a6767bd2468d076c3c00ac"
server_key_iv: "bd6a698941753df92d4be142daf2f62f"
server_key_version: "encryptionkey@0"
I20260812 07:07:55.975479 1247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "bda99d14cdea44a493b660c86ab3282e"
format_stamp: "Formatted at 2026-08-12 07:07:55 on dist-test-slave-csg5"
server_key: "5ab1685de7a6767bd2468d076c3c00ac"
server_key_iv: "bd6a698941753df92d4be142daf2f62f"
server_key_version: "encryptionkey@0"
I20260812 07:07:55.980050 1247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 07:07:55.983469 1266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:55.985797 1247 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260812 07:07:55.986011 1247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "bda99d14cdea44a493b660c86ab3282e"
format_stamp: "Formatted at 2026-08-12 07:07:55 on dist-test-slave-csg5"
server_key: "5ab1685de7a6767bd2468d076c3c00ac"
server_key_iv: "bd6a698941753df92d4be142daf2f62f"
server_key_version: "encryptionkey@0"
I20260812 07:07:55.986174 1247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:56.002595 1247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:56.006358 1247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:56.006596 1247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:56.008206 1247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:56.008297 1247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:56.008373 1247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:56.008425 1247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:56.020784 1247 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:42487
Aug 12 07:07:56 dist-test-slave-csg5 krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518475, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:56.020865 1395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:42487 every 8 connection(s)
I20260812 07:07:56.022019 1247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:07:56.026202 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 1247
I20260812 07:07:56.026348 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:07:56.026494 1220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.009407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:42625 (local address 127.31.156.126:44441)
0812 07:07:56.009582 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.009588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.010690 (+ 1102us) server_negotiation.cc:407] Connection header received
0812 07:07:56.012091 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.012095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.012167 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.012286 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:56.014728 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.015770 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.017241 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.017500 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.017595 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.021647 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:56.021651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:56.021654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:56.021690 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:56.023813 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.024487 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.024490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.024491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.024592 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.024954 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.024956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.024957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.025259 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:56.025393 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.026009 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.026277 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:07:56.026806 32369 external_mini_cluster.cc:1428] Setting key 709b4277cd8c5c51f86ca72d46162a86
I20260812 07:07:56.027446 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.009920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:44441 (local address 127.31.156.65:42625)
0812 07:07:56.010465 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:56.010526 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:56.011831 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:56.012509 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:56.012524 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:56.013164 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:56.014441 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.014476 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.015940 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.015944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.017007 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.017052 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.017710 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.017714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.017953 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.019082 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:56.019114 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:56.021380 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:56.023954 (+ 2574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.023960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.023972 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.024258 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.024691 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.024693 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.024695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.024848 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.025431 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:56.025436 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:56.025695 (+ 259us) client_negotiation.cc:769] Sending connection context
0812 07:07:56.025877 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 07:07:56.026176 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":167,"threads_started":1}
I20260812 07:07:56.029075 1396 heartbeater.cc:347] Connected to a master server at 127.31.156.126:44441
I20260812 07:07:56.029277 1396 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:56.029966 1396 heartbeater.cc:511] Master 127.31.156.126:44441 requested a full tablet report, sending...
I20260812 07:07:56.031888 1163 ts_manager.cc:194] Registered new tserver with Master: bda99d14cdea44a493b660c86ab3282e (127.31.156.65:42487)
I20260812 07:07:56.033434 1163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:42625
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:07:56.042260 1220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.032196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:42339 (local address 127.31.156.126:44441)
0812 07:07:56.032346 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.032351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.032653 (+ 302us) server_negotiation.cc:407] Connection header received
0812 07:07:56.032817 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.032821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.032868 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.032975 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:56.033508 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.034434 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.036189 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.036425 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.036517 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.038144 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:56.038146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:56.038148 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:56.038178 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:56.039774 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.040405 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.040409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.040410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.040511 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.041023 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.041026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.041028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.041316 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:56.041540 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.041861 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.042029 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:07:56.042331 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.032143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:44441 (local address 127.31.156.65:42339)
0812 07:07:56.032547 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:56.032560 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:56.032704 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:56.032996 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:56.032999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:56.033252 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:56.033371 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.033378 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.034620 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.034627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.036023 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.036042 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.036568 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.036575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.036720 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.037302 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:56.037325 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:56.037966 (+ 641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:56.039905 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.039909 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.039911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.040243 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.040644 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.040651 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.040656 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.040814 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.041521 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:56.041524 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:56.041661 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 07:07:56.041794 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 07:07:56.042044 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":31,"thread_start_us":135,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:07:56.067945 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:44441
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:56.190377 1406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:56.190759 1406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:56.190860 1406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:56.195389 1406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:56.195499 1406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:56.195580 1406 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:07:56.195665 1406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:07:56.201500 1406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.156.126:44441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:56.202781 1406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:56.204156 1406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:07:56.215257 1406 server_base.cc:1034] running on GCE node
W20260812 07:07:56.215327 1414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:56.215200 1413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:56.215202 1416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:56.216097 1406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:56.216876 1406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:56.218086 1406 hybrid_clock.cc:648] HybridClock initialized: now 1786518476218029 us; error 73 us; skew 500 ppm
Aug 12 07:07:56 dist-test-slave-csg5 krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518476, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:56.221397 1406 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:07:56.222435 1406 webserver.cc:460] Webserver started at http://127.31.156.66:39303/ using document root <none> and password file <none>
I20260812 07:07:56.223107 1406 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:56.223181 1406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:56.223407 1406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:56.225782 1406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "e290acb36ebe4464b39105dbed2f435e"
format_stamp: "Formatted at 2026-08-12 07:07:56 on dist-test-slave-csg5"
server_key: "6b0f8dd0a54672b4300d1c01fd2ef183"
server_key_iv: "6a4536b54481a9adab8160beb6d4958f"
server_key_version: "encryptionkey@0"
I20260812 07:07:56.226526 1406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "e290acb36ebe4464b39105dbed2f435e"
format_stamp: "Formatted at 2026-08-12 07:07:56 on dist-test-slave-csg5"
server_key: "6b0f8dd0a54672b4300d1c01fd2ef183"
server_key_iv: "6a4536b54481a9adab8160beb6d4958f"
server_key_version: "encryptionkey@0"
I20260812 07:07:56.231370 1406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:07:56.234277 1426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:56.235792 1406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:07:56.235976 1406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "e290acb36ebe4464b39105dbed2f435e"
format_stamp: "Formatted at 2026-08-12 07:07:56 on dist-test-slave-csg5"
server_key: "6b0f8dd0a54672b4300d1c01fd2ef183"
server_key_iv: "6a4536b54481a9adab8160beb6d4958f"
server_key_version: "encryptionkey@0"
I20260812 07:07:56.236142 1406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:56.252866 1406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:56.257630 1406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:56.257892 1406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:56.259663 1406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:56.259733 1406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:56.259807 1406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:56.259842 1406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:56.271893 1406 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:46721
Aug 12 07:07:56 dist-test-slave-csg5 krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518476, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:56.271970 1556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:46721 every 8 connection(s)
I20260812 07:07:56.272897 1406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:07:56.277100 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 1406
I20260812 07:07:56.277223 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:07:56.277366 1220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.260920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:46207 (local address 127.31.156.126:44441)
0812 07:07:56.261067 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.261071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.262331 (+ 1260us) server_negotiation.cc:407] Connection header received
0812 07:07:56.263934 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.263938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.264001 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.264100 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:56.265975 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.266924 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.268838 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.269077 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.269172 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.272804 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:56.272808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:56.272811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:56.272844 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:56.274640 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.275210 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.275213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.275214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.275408 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.275860 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.275863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.275865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.276095 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:56.276204 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.276912 (+ 708us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.277079 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:07:56.277561 32369 external_mini_cluster.cc:1428] Setting key 4125a7fa8f6c589e1a27362bd704dba9
I20260812 07:07:56.278276 1435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.261533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:44441 (local address 127.31.156.66:46207)
0812 07:07:56.262142 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:56.262197 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:56.263647 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:56.264436 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:56.264444 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:56.264991 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:56.265805 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.265827 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.267067 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.267074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.268685 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.268698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.269307 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.269316 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.269581 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.270427 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:56.270458 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:56.272544 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:56.274818 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.274823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.274835 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.275108 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.275532 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.275534 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.275536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.275709 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.276215 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:56.276221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:56.276552 (+ 331us) client_negotiation.cc:769] Sending connection context
0812 07:07:56.276727 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 07:07:56.277055 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":156,"threads_started":1}
I20260812 07:07:56.279743 1557 heartbeater.cc:347] Connected to a master server at 127.31.156.126:44441
I20260812 07:07:56.279994 1557 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:56.280767 1557 heartbeater.cc:511] Master 127.31.156.126:44441 requested a full tablet report, sending...
I20260812 07:07:56.282239 1163 ts_manager.cc:194] Registered new tserver with Master: e290acb36ebe4464b39105dbed2f435e (127.31.156.66:46721)
I20260812 07:07:56.283140 1163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:46207
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:07:56.292891 1220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.282102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:51085 (local address 127.31.156.126:44441)
0812 07:07:56.282249 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.282257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.282887 (+ 630us) server_negotiation.cc:407] Connection header received
0812 07:07:56.282994 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.282999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.283059 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.283155 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:56.283861 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.284694 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.285838 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.286257 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.286386 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.288066 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:56.288073 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:56.288079 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:56.288117 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:56.290231 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.290899 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.290903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.290905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.291009 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.291565 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.291570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.291572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.291869 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:56.291964 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.292608 (+ 644us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.292743 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:07:56.292884 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.282366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:44441 (local address 127.31.156.66:51085)
0812 07:07:56.282766 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:56.282783 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:56.282905 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:56.283200 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:56.283203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:56.283516 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:56.283694 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.283702 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.284872 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.284876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.285654 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.285665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.286446 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.286449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.286526 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.287459 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:56.287472 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:56.287873 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:56.290430 (+ 2557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.290437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.290441 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.290778 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.291120 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.291122 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.291123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.291372 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.292002 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:56.292008 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:56.292115 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 07:07:56.292287 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 07:07:56.292526 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:07:56.319658 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:44441
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:56.448238 1567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:56.448596 1567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:56.448647 1567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:56.452977 1567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:56.453056 1567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:56.453099 1567 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:07:56.453173 1567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:07:56.459071 1567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.156.126:44441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:56.460323 1567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:56.461469 1567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:56.472409 1573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:56.472651 1572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:56.472685 1575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:56.473093 1567 server_base.cc:1034] running on GCE node
I20260812 07:07:56.473655 1567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:56.474448 1567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:56.475777 1567 hybrid_clock.cc:648] HybridClock initialized: now 1786518476475744 us; error 54 us; skew 500 ppm
Aug 12 07:07:56 dist-test-slave-csg5 krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518476, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:56.479722 1567 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:07:56.481159 1567 webserver.cc:460] Webserver started at http://127.31.156.67:35695/ using document root <none> and password file <none>
I20260812 07:07:56.481827 1567 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:56.481900 1567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:56.482103 1567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:56.484719 1567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "4ee9bbfc5b254308832fc75e6b32a895"
format_stamp: "Formatted at 2026-08-12 07:07:56 on dist-test-slave-csg5"
server_key: "5ac18dce502de8b29bcc09e830157392"
server_key_iv: "4aa887525a785c4c70e254d5f2c63e00"
server_key_version: "encryptionkey@0"
I20260812 07:07:56.485426 1567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "4ee9bbfc5b254308832fc75e6b32a895"
format_stamp: "Formatted at 2026-08-12 07:07:56 on dist-test-slave-csg5"
server_key: "5ac18dce502de8b29bcc09e830157392"
server_key_iv: "4aa887525a785c4c70e254d5f2c63e00"
server_key_version: "encryptionkey@0"
I20260812 07:07:56.490253 1567 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:07:56.493402 1582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:56.495091 1567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:07:56.495251 1567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "4ee9bbfc5b254308832fc75e6b32a895"
format_stamp: "Formatted at 2026-08-12 07:07:56 on dist-test-slave-csg5"
server_key: "5ac18dce502de8b29bcc09e830157392"
server_key_iv: "4aa887525a785c4c70e254d5f2c63e00"
server_key_version: "encryptionkey@0"
I20260812 07:07:56.495384 1567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:56.512681 1567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:56.516628 1567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:56.516870 1567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:56.518615 1567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:56.518728 1567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:56.518843 1567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:56.518884 1567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:07:56 dist-test-slave-csg5 krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518476, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:56.531486 1567 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:40093
I20260812 07:07:56.531587 1709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:40093 every 8 connection(s)
I20260812 07:07:56.532472 1567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:07:56.535568 1220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.519828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:46343 (local address 127.31.156.126:44441)
0812 07:07:56.520006 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.520013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.520981 (+ 968us) server_negotiation.cc:407] Connection header received
0812 07:07:56.522255 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.522259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.522332 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.522453 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:56.524127 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.524728 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.526054 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.526306 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.526442 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.530783 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:56.530787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:56.530790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:56.530823 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:56.532783 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.533495 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.533499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.533501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.533600 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.534032 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.534035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.534036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.534268 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:56.534360 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.535276 (+ 916us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.535426 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:07:56.536404 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.520232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:44441 (local address 127.31.156.67:46343)
0812 07:07:56.520774 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:56.520844 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:56.521999 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:56.522728 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:56.522745 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:56.523229 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:56.523914 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.523942 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.524902 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.524906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.525895 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.525906 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.526506 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.526512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.526799 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.527560 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:56.527586 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:56.530521 (+ 2935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:56.532929 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.532936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.532954 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.533371 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.533745 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.533751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.533756 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.533939 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.534513 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:56.534519 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:56.534889 (+ 370us) client_negotiation.cc:769] Sending connection context
0812 07:07:56.535071 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 07:07:56.535377 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":154,"threads_started":1}
I20260812 07:07:56.537918 1710 heartbeater.cc:347] Connected to a master server at 127.31.156.126:44441
I20260812 07:07:56.538161 1710 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:56.538901 1710 heartbeater.cc:511] Master 127.31.156.126:44441 requested a full tablet report, sending...
I20260812 07:07:56.539191 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 1567
I20260812 07:07:56.539444 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:07:56.539816 32369 external_mini_cluster.cc:1428] Setting key 70eba7e47a07c298b1e623c21a3f59b8
I20260812 07:07:56.540151 1163 ts_manager.cc:194] Registered new tserver with Master: 4ee9bbfc5b254308832fc75e6b32a895 (127.31.156.67:40093)
I20260812 07:07:56.541033 1163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:46343
I20260812 07:07:56.542443 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:07:56.549542 1220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.540706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:34157 (local address 127.31.156.126:44441)
0812 07:07:56.540853 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.540858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.541188 (+ 330us) server_negotiation.cc:407] Connection header received
0812 07:07:56.541418 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.541421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.541473 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.541559 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:56.542241 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.542980 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.543936 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.544181 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.544297 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.545541 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:56.545547 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:56.545551 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:56.545585 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:56.547325 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.547956 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.547959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.547960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.548047 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.548451 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.548457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.548458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.548672 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:56.548821 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.549302 (+ 481us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.549415 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:07:56.549651 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.540653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:44441 (local address 127.31.156.67:34157)
0812 07:07:56.541106 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:56.541122 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:56.541300 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:56.541680 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:56.541683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:56.541969 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:56.542122 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.542131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.543119 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.543123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.543804 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.543814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.544307 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.544311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.544406 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.545009 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:56.545021 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:56.545388 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:56.547472 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.547479 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.547485 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.547812 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.548156 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:56.548163 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:56.548169 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:56.548320 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:56.548848 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:56.548854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:56.549006 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 07:07:56.549152 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 07:07:56.549380 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":12,"thread_start_us":124,"threads_started":1}
Aug 12 07:07:56 dist-test-slave-csg5 krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518476, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:07:56 dist-test-slave-csg5 krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518476, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:56.575096 1220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.565179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38162 (local address 127.31.156.126:44441)
0812 07:07:56.565332 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.565338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.565356 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:07:56.565409 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.565411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.565457 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.565546 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:56.566462 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.567147 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.568129 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.568475 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.568583 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.571601 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:56.571605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:56.571607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:56.571638 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:56.573332 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.573770 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.573773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.573774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.573856 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:56.574131 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:56.574135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:56.574136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:56.574382 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:56.574507 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.574828 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.574956 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:07:56.578877 1163 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:07:56.582412 1163 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:07:56.599947 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.593475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.31.156.65:42487)
0812 07:07:56.593790 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.593799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.593828 (+ 29us) server_negotiation.cc:407] Connection header received
0812 07:07:56.593918 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.593924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.594840 (+ 916us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.595006 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:56.595269 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.596440 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.598064 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.599036 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.599258 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.599441 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.599641 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.599764 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":112,"threads_started":1}
I20260812 07:07:56.600003 1734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.592188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:42487 (local address 127.0.0.1:43772)
0812 07:07:56.592804 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:56.592907 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:56.593058 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:56.594997 (+ 1939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:56.595004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:56.595026 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:56.595138 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.595145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.596564 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.596568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.597915 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.597936 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.599203 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.599207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.599394 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.599449 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 07:07:56.599579 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 07:07:56.599782 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":57,"thread_start_us":103,"threads_started":1}
I20260812 07:07:56.600003 1735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.592361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:40093 (local address 127.0.0.1:53492)
0812 07:07:56.593366 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:56.593378 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:56.593498 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:56.594422 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:56.594429 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:56.594454 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:56.594649 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.594697 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.596325 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.596328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.598035 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.598048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.599203 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.599207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.599353 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.599446 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 07:07:56.599582 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 07:07:56.599782 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":949,"mutex_wait_us":163,"thread_start_us":162,"threads_started":1}
I20260812 07:07:56.600315 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.593022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.31.156.67:40093)
0812 07:07:56.593959 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.593970 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.593991 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:07:56.594061 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.594065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.594234 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.594428 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:56.594827 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.596186 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.598154 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.599036 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.599766 (+ 730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.599950 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.600010 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.600135 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":131,"threads_started":1}
I20260812 07:07:56.600942 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.593031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:46721 (local address 127.0.0.1:44778)
0812 07:07:56.593961 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:56.593976 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:56.594064 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:56.594807 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:56.594814 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:56.594832 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:56.594913 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.594917 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.596093 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.596100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.598035 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.598048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.599119 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.599123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.600567 (+ 1444us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.600603 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:07:56.600697 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 07:07:56.600793 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":243,"thread_start_us":110,"threads_started":1}
I20260812 07:07:56.601213 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.593251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44778 (local address 127.31.156.66:46721)
0812 07:07:56.593653 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.593668 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.594178 (+ 510us) server_negotiation.cc:407] Connection header received
0812 07:07:56.594253 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.594264 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.594469 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.594626 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:56.595073 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.595959 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.598202 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.598975 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.599118 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.599335 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.600816 (+ 1481us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.600929 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":137,"threads_started":1}
I20260812 07:07:56.603729 1637 tablet_service.cc:1467] Processing CreateTablet for tablet 2f1bb470e36c48209af8bbf4246f9c71 (DEFAULT_TABLE table=test-table [id=30a8845e4c71480d8a5001e9c1a53ae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:56.604162 1481 tablet_service.cc:1467] Processing CreateTablet for tablet 2f1bb470e36c48209af8bbf4246f9c71 (DEFAULT_TABLE table=test-table [id=30a8845e4c71480d8a5001e9c1a53ae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:56.604162 1319 tablet_service.cc:1467] Processing CreateTablet for tablet 2f1bb470e36c48209af8bbf4246f9c71 (DEFAULT_TABLE table=test-table [id=30a8845e4c71480d8a5001e9c1a53ae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:56.604991 1637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f1bb470e36c48209af8bbf4246f9c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:56.605273 1319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f1bb470e36c48209af8bbf4246f9c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:56.605273 1481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f1bb470e36c48209af8bbf4246f9c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:56.613723 1740 tablet_bootstrap.cc:492] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e: Bootstrap starting.
I20260812 07:07:56.615068 1739 tablet_bootstrap.cc:492] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e: Bootstrap starting.
I20260812 07:07:56.615393 1741 tablet_bootstrap.cc:492] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895: Bootstrap starting.
I20260812 07:07:56.616721 1740 tablet_bootstrap.cc:654] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:56.617432 1741 tablet_bootstrap.cc:654] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:56.617434 1739 tablet_bootstrap.cc:654] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:56.617813 1740 log.cc:824] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:56.618263 1741 log.cc:824] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:56.618350 1739 log.cc:824] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:56.620213 1740 tablet_bootstrap.cc:492] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e: No bootstrap required, opened a new log
I20260812 07:07:56.620246 1741 tablet_bootstrap.cc:492] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895: No bootstrap required, opened a new log
I20260812 07:07:56.620462 1741 ts_tablet_manager.cc:1397] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 07:07:56.620458 1740 ts_tablet_manager.cc:1397] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:07:56.622794 1739 tablet_bootstrap.cc:492] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e: No bootstrap required, opened a new log
I20260812 07:07:56.623103 1739 ts_tablet_manager.cc:1397] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:07:56.624234 1741 raft_consensus.cc:348] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } }
I20260812 07:07:56.624521 1741 raft_consensus.cc:374] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:56.624583 1741 raft_consensus.cc:729] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee9bbfc5b254308832fc75e6b32a895, State: Initialized, Role: FOLLOWER
I20260812 07:07:56.625118 1741 consensus_queue.cc:260] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } }
I20260812 07:07:56.624997 1740 raft_consensus.cc:348] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } }
I20260812 07:07:56.625298 1740 raft_consensus.cc:374] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:56.625370 1740 raft_consensus.cc:729] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e290acb36ebe4464b39105dbed2f435e, State: Initialized, Role: FOLLOWER
I20260812 07:07:56.625952 1710 heartbeater.cc:503] Master 127.31.156.126:44441 was elected leader, sending a full tablet report...
I20260812 07:07:56.626340 1741 ts_tablet_manager.cc:1428] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260812 07:07:56.627653 1740 consensus_queue.cc:260] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } }
I20260812 07:07:56.628466 1739 raft_consensus.cc:348] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } }
I20260812 07:07:56.628780 1739 raft_consensus.cc:374] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:56.628854 1739 raft_consensus.cc:729] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda99d14cdea44a493b660c86ab3282e, State: Initialized, Role: FOLLOWER
I20260812 07:07:56.628965 1740 ts_tablet_manager.cc:1428] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 07:07:56.629410 1739 consensus_queue.cc:260] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } }
I20260812 07:07:56.629482 1557 heartbeater.cc:503] Master 127.31.156.126:44441 was elected leader, sending a full tablet report...
I20260812 07:07:56.630554 1739 ts_tablet_manager.cc:1428] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 07:07:56.630977 1396 heartbeater.cc:503] Master 127.31.156.126:44441 was elected leader, sending a full tablet report...
I20260812 07:07:56.654151 1747 raft_consensus.cc:482] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:56.654511 1747 raft_consensus.cc:504] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } }
I20260812 07:07:56.655766 1747 leader_election.cc:288] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee9bbfc5b254308832fc75e6b32a895 (127.31.156.67:40093), e290acb36ebe4464b39105dbed2f435e (127.31.156.66:46721)
I20260812 07:07:56.661799 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.656799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:44705 (local address 127.31.156.66:46721)
0812 07:07:56.656930 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.656936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.657104 (+ 168us) server_negotiation.cc:407] Connection header received
0812 07:07:56.657207 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.657210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.657255 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.657359 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:56.657616 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.658374 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.660434 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.661072 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.661210 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.661269 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.661563 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.661666 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:07:56.661824 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.656289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:43491 (local address 127.31.156.67:40093)
0812 07:07:56.656443 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.656447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.656544 (+ 97us) server_negotiation.cc:407] Connection header received
0812 07:07:56.656699 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.656702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.656758 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.656845 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:56.657364 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.658168 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.660448 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.661072 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.661289 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.661343 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.661563 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.661715 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:07:56.661795 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.656074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:40093 (local address 127.31.156.65:43491)
0812 07:07:56.656432 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:56.656451 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:56.656591 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:56.656960 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:56.656964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:56.656985 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:56.657157 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.657169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.658334 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.658339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.660299 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.660314 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.661290 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.661293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.661388 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.661412 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:07:56.661491 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 07:07:56.661570 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":213,"thread_start_us":168,"threads_started":1}
I20260812 07:07:56.661799 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.656643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:46721 (local address 127.31.156.65:44705)
0812 07:07:56.657003 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:56.657017 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:56.657125 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:56.657389 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:56.657391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:56.657409 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:56.657500 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.657505 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.658535 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.658540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.660320 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:56.660327 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.661230 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.661235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:56.661345 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.661377 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:07:56.661464 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 07:07:56.661570 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":265,"thread_start_us":122,"threads_started":1}
I20260812 07:07:56.662775 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f1bb470e36c48209af8bbf4246f9c71" candidate_uuid: "bda99d14cdea44a493b660c86ab3282e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee9bbfc5b254308832fc75e6b32a895" is_pre_election: true
I20260812 07:07:56.663103 1503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f1bb470e36c48209af8bbf4246f9c71" candidate_uuid: "bda99d14cdea44a493b660c86ab3282e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e290acb36ebe4464b39105dbed2f435e" is_pre_election: true
I20260812 07:07:56.663182 1663 raft_consensus.cc:2454] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda99d14cdea44a493b660c86ab3282e in term 0.
I20260812 07:07:56.663410 1503 raft_consensus.cc:2454] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda99d14cdea44a493b660c86ab3282e in term 0.
I20260812 07:07:56.663758 1267 leader_election.cc:302] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee9bbfc5b254308832fc75e6b32a895, bda99d14cdea44a493b660c86ab3282e; no voters:
I20260812 07:07:56.664135 1747 raft_consensus.cc:2781] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:07:56.664256 1747 raft_consensus.cc:482] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:07:56.664336 1747 raft_consensus.cc:3037] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:56.665508 1747 raft_consensus.cc:504] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } }
I20260812 07:07:56.665966 1747 leader_election.cc:288] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [CANDIDATE]: Term 1 election: Requested vote from peers 4ee9bbfc5b254308832fc75e6b32a895 (127.31.156.67:40093), e290acb36ebe4464b39105dbed2f435e (127.31.156.66:46721)
I20260812 07:07:56.666476 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f1bb470e36c48209af8bbf4246f9c71" candidate_uuid: "bda99d14cdea44a493b660c86ab3282e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee9bbfc5b254308832fc75e6b32a895"
I20260812 07:07:56.666541 1503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f1bb470e36c48209af8bbf4246f9c71" candidate_uuid: "bda99d14cdea44a493b660c86ab3282e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e290acb36ebe4464b39105dbed2f435e"
I20260812 07:07:56.666677 1663 raft_consensus.cc:3037] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:56.666742 1503 raft_consensus.cc:3037] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:56.668227 1663 raft_consensus.cc:2454] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda99d14cdea44a493b660c86ab3282e in term 1.
I20260812 07:07:56.668581 1503 raft_consensus.cc:2454] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda99d14cdea44a493b660c86ab3282e in term 1.
I20260812 07:07:56.668579 1267 leader_election.cc:302] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee9bbfc5b254308832fc75e6b32a895, bda99d14cdea44a493b660c86ab3282e; no voters:
I20260812 07:07:56.668787 1747 raft_consensus.cc:2781] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:56.669091 1747 raft_consensus.cc:686] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 1 LEADER]: Becoming Leader. State: Replica: bda99d14cdea44a493b660c86ab3282e, State: Running, Role: LEADER
I20260812 07:07:56.669566 1747 consensus_queue.cc:237] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } }
I20260812 07:07:56.673550 1160 catalog_manager.cc:5495] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e reported cstate change: term changed from 0 to 1, leader changed from <none> to bda99d14cdea44a493b660c86ab3282e (127.31.156.65). New cstate: current_term: 1 leader_uuid: "bda99d14cdea44a493b660c86ab3282e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } health_report { overall_health: UNKNOWN } } }
I20260812 07:07:56.702062 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:56.697134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.31.156.65:42487)
0812 07:07:56.697283 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:56.697290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:56.697344 (+ 54us) server_negotiation.cc:407] Connection header received
0812 07:07:56.697629 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:56.697632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:56.697691 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:56.697803 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:07:56.698101 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.698707 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.699979 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:56.700417 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:56.700563 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:56.700997 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:07:56.701521 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:07:56.701628 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:56.701787 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 07:07:56.701853 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:07:56.709438 1503 raft_consensus.cc:1261] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e [term 1 FOLLOWER]: Refusing update from remote peer bda99d14cdea44a493b660c86ab3282e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:56.709771 1663 raft_consensus.cc:1261] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895 [term 1 FOLLOWER]: Refusing update from remote peer bda99d14cdea44a493b660c86ab3282e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:56.710501 1747 consensus_queue.cc:1035] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:56.710789 1750 consensus_queue.cc:1035] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:56.720422 1756 mvcc.cc:205] Tried to move back new op lower bound from 7317579680593133568 to 7317579680441565184. Current Snapshot: MvccSnapshot[applied={T|T < 7317579680593133568}]
I20260812 07:07:56.720715 1757 mvcc.cc:205] Tried to move back new op lower bound from 7317579680593133568 to 7317579680441565184. Current Snapshot: MvccSnapshot[applied={T|T < 7317579680593133568}]
I20260812 07:07:56.736640 1160 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:56.736881 1160 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:56.740039 1160 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae: Sending DeleteTablet for 3 replicas of tablet 2f1bb470e36c48209af8bbf4246f9c71
I20260812 07:07:56.741097 1319 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f1bb470e36c48209af8bbf4246f9c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:56 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:43772
I20260812 07:07:56.741074 1481 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f1bb470e36c48209af8bbf4246f9c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:56 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:44778
I20260812 07:07:56.741080 1637 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f1bb470e36c48209af8bbf4246f9c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:56 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:53492
I20260812 07:07:56.741644 1766 tablet_replica.cc:330] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895: stopping tablet replica
I20260812 07:07:56.741644 1765 tablet_replica.cc:330] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e: stopping tablet replica
I20260812 07:07:56.742034 1766 raft_consensus.cc:2229] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:56.742038 1765 raft_consensus.cc:2229] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:56.742419 1766 raft_consensus.cc:2258] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:56.742419 1765 raft_consensus.cc:2258] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:56.743561 1160 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:07:56.744086 1160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:07:56.745425 1766 ts_tablet_manager.cc:1905] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:56.747886 1765 ts_tablet_manager.cc:1905] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:56.749894 1481 tablet_service.cc:1467] Processing CreateTablet for tablet 0205cb91b8ee43caaac08d953cc6e1e1 (DEFAULT_TABLE table=test-table [id=c03743d73cd94ace9d9430543c00776a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:56.750192 1481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0205cb91b8ee43caaac08d953cc6e1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:56.750164 1319 tablet_service.cc:1467] Processing CreateTablet for tablet 0205cb91b8ee43caaac08d953cc6e1e1 (DEFAULT_TABLE table=test-table [id=c03743d73cd94ace9d9430543c00776a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:56.750458 1319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0205cb91b8ee43caaac08d953cc6e1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:56.751016 1637 tablet_service.cc:1467] Processing CreateTablet for tablet 0205cb91b8ee43caaac08d953cc6e1e1 (DEFAULT_TABLE table=test-table [id=c03743d73cd94ace9d9430543c00776a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:56.751120 1765 ts_tablet_manager.cc:1918] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:56.751214 1765 log.cc:1196] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/2f1bb470e36c48209af8bbf4246f9c71
I20260812 07:07:56.751319 1637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0205cb91b8ee43caaac08d953cc6e1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:56.751564 1765 ts_tablet_manager.cc:1939] T 2f1bb470e36c48209af8bbf4246f9c71 P e290acb36ebe4464b39105dbed2f435e: Deleting consensus metadata
I20260812 07:07:56.752627 1740 tablet_bootstrap.cc:492] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e: Bootstrap starting.
I20260812 07:07:56.753512 1766 ts_tablet_manager.cc:1918] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:56.753612 1766 log.cc:1196] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/2f1bb470e36c48209af8bbf4246f9c71
I20260812 07:07:56.753691 1740 tablet_bootstrap.cc:654] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:56.753747 1741 tablet_bootstrap.cc:492] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895: Bootstrap starting.
I20260812 07:07:56.754766 1741 tablet_bootstrap.cc:654] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:56.754886 1767 tablet_replica.cc:330] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e: stopping tablet replica
I20260812 07:07:56.755334 1767 raft_consensus.cc:2229] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:07:56.755618 1766 ts_tablet_manager.cc:1939] T 2f1bb470e36c48209af8bbf4246f9c71 P 4ee9bbfc5b254308832fc75e6b32a895: Deleting consensus metadata
I20260812 07:07:56.755825 1740 tablet_bootstrap.cc:492] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e: No bootstrap required, opened a new log
I20260812 07:07:56.755837 1767 raft_consensus.cc:2258] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:56.755914 1740 ts_tablet_manager.cc:1397] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:07:56.756371 1740 raft_consensus.cc:348] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:56.756484 1740 raft_consensus.cc:374] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:56.756513 1740 raft_consensus.cc:729] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e290acb36ebe4464b39105dbed2f435e, State: Initialized, Role: FOLLOWER
I20260812 07:07:56.756629 1740 consensus_queue.cc:260] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:56.756933 1740 ts_tablet_manager.cc:1428] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:07:56.757030 1145 catalog_manager.cc:4841] TS 4ee9bbfc5b254308832fc75e6b32a895 (127.31.156.67:40093): tablet 2f1bb470e36c48209af8bbf4246f9c71 (table test-table [id=30a8845e4c71480d8a5001e9c1a53ae8]) successfully deleted
I20260812 07:07:56.757922 1739 tablet_bootstrap.cc:492] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e: Bootstrap starting.
I20260812 07:07:56.758280 1145 catalog_manager.cc:4841] TS e290acb36ebe4464b39105dbed2f435e (127.31.156.66:46721): tablet 2f1bb470e36c48209af8bbf4246f9c71 (table test-table [id=30a8845e4c71480d8a5001e9c1a53ae8]) successfully deleted
I20260812 07:07:56.758322 1767 ts_tablet_manager.cc:1905] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:56.758930 1739 tablet_bootstrap.cc:654] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:56.759532 1741 tablet_bootstrap.cc:492] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895: No bootstrap required, opened a new log
I20260812 07:07:56.759660 1741 ts_tablet_manager.cc:1397] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260812 07:07:56.760179 1741 raft_consensus.cc:348] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:56.760329 1741 raft_consensus.cc:374] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:56.760370 1741 raft_consensus.cc:729] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee9bbfc5b254308832fc75e6b32a895, State: Initialized, Role: FOLLOWER
I20260812 07:07:56.760599 1741 consensus_queue.cc:260] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:56.760839 1739 tablet_bootstrap.cc:492] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e: No bootstrap required, opened a new log
I20260812 07:07:56.760927 1741 ts_tablet_manager.cc:1428] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 07:07:56.760936 1739 ts_tablet_manager.cc:1397] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:07:56.761360 1767 ts_tablet_manager.cc:1918] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:56.761471 1767 log.cc:1196] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/wal/wals/2f1bb470e36c48209af8bbf4246f9c71
I20260812 07:07:56.761415 1739 raft_consensus.cc:348] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:56.761543 1739 raft_consensus.cc:374] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:56.761587 1739 raft_consensus.cc:729] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda99d14cdea44a493b660c86ab3282e, State: Initialized, Role: FOLLOWER
I20260812 07:07:56.761750 1739 consensus_queue.cc:260] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:56.761817 1767 ts_tablet_manager.cc:1939] T 2f1bb470e36c48209af8bbf4246f9c71 P bda99d14cdea44a493b660c86ab3282e: Deleting consensus metadata
I20260812 07:07:56.762079 1739 ts_tablet_manager.cc:1428] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:07:56.763675 1150 catalog_manager.cc:4841] TS bda99d14cdea44a493b660c86ab3282e (127.31.156.65:42487): tablet 2f1bb470e36c48209af8bbf4246f9c71 (table test-table [id=30a8845e4c71480d8a5001e9c1a53ae8]) successfully deleted
W20260812 07:07:56.773952 1397 tablet.cc:2369] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:07:56.774237 1558 tablet.cc:2369] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:07:56.783736 1711 tablet.cc:2369] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:07:57.020694 1747 raft_consensus.cc:482] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:57.020897 1747 raft_consensus.cc:504] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.021464 1747 leader_election.cc:288] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e290acb36ebe4464b39105dbed2f435e (127.31.156.66:46721), 4ee9bbfc5b254308832fc75e6b32a895 (127.31.156.67:40093)
I20260812 07:07:57.022023 1503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0205cb91b8ee43caaac08d953cc6e1e1" candidate_uuid: "bda99d14cdea44a493b660c86ab3282e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e290acb36ebe4464b39105dbed2f435e" is_pre_election: true
I20260812 07:07:57.022070 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0205cb91b8ee43caaac08d953cc6e1e1" candidate_uuid: "bda99d14cdea44a493b660c86ab3282e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee9bbfc5b254308832fc75e6b32a895" is_pre_election: true
I20260812 07:07:57.022197 1503 raft_consensus.cc:2454] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda99d14cdea44a493b660c86ab3282e in term 0.
I20260812 07:07:57.022336 1663 raft_consensus.cc:2454] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda99d14cdea44a493b660c86ab3282e in term 0.
I20260812 07:07:57.022558 1267 leader_election.cc:302] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bda99d14cdea44a493b660c86ab3282e, e290acb36ebe4464b39105dbed2f435e; no voters:
I20260812 07:07:57.022773 1747 raft_consensus.cc:2781] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:07:57.022836 1747 raft_consensus.cc:482] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:07:57.022876 1747 raft_consensus.cc:3037] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:57.023973 1747 raft_consensus.cc:504] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.024367 1747 leader_election.cc:288] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [CANDIDATE]: Term 1 election: Requested vote from peers e290acb36ebe4464b39105dbed2f435e (127.31.156.66:46721), 4ee9bbfc5b254308832fc75e6b32a895 (127.31.156.67:40093)
I20260812 07:07:57.024789 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0205cb91b8ee43caaac08d953cc6e1e1" candidate_uuid: "bda99d14cdea44a493b660c86ab3282e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee9bbfc5b254308832fc75e6b32a895"
I20260812 07:07:57.024798 1503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0205cb91b8ee43caaac08d953cc6e1e1" candidate_uuid: "bda99d14cdea44a493b660c86ab3282e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e290acb36ebe4464b39105dbed2f435e"
I20260812 07:07:57.024930 1663 raft_consensus.cc:3037] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:57.024940 1503 raft_consensus.cc:3037] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:57.025941 1503 raft_consensus.cc:2454] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda99d14cdea44a493b660c86ab3282e in term 1.
I20260812 07:07:57.026089 1663 raft_consensus.cc:2454] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda99d14cdea44a493b660c86ab3282e in term 1.
I20260812 07:07:57.026338 1267 leader_election.cc:302] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bda99d14cdea44a493b660c86ab3282e, e290acb36ebe4464b39105dbed2f435e; no voters:
I20260812 07:07:57.026517 1747 raft_consensus.cc:2781] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:57.026608 1747 raft_consensus.cc:686] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 1 LEADER]: Becoming Leader. State: Replica: bda99d14cdea44a493b660c86ab3282e, State: Running, Role: LEADER
I20260812 07:07:57.026829 1747 consensus_queue.cc:237] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.028913 1160 catalog_manager.cc:5495] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e reported cstate change: term changed from 0 to 1, leader changed from <none> to bda99d14cdea44a493b660c86ab3282e (127.31.156.65). New cstate: current_term: 1 leader_uuid: "bda99d14cdea44a493b660c86ab3282e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } health_report { overall_health: UNKNOWN } } }
Aug 12 07:07:57 dist-test-slave-csg5 krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518475, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:57.087419 1779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.077030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:44441 (local address 127.0.0.1:38164)
0812 07:07:57.077325 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:57.077342 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:57.077471 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:57.077749 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:57.077751 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:57.078154 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:57.078337 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.078344 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.079376 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.079380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.079996 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.080004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.080775 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.080778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.080852 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.081536 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:57.081549 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:57.083528 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:57.085453 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:57.085467 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:57.085475 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:57.085741 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:57.086061 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:57.086065 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:57.086067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:57.086157 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:57.086706 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:57.086714 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:57.086849 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 07:07:57.086974 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 07:07:57.087183 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":4,"thread_start_us":78,"threads_started":1}
I20260812 07:07:57.087430 1780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.077176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38164 (local address 127.31.156.126:44441)
0812 07:07:57.077467 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:57.077474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:57.077492 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:07:57.077537 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:57.077539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:57.077579 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:57.077667 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:57.078472 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.079189 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.080318 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.080605 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.080770 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.083704 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:57.083707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:57.083734 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:57.083757 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:57.085326 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:57.085848 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:57.085853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:57.085855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:57.085942 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:57.086361 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:57.086364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:57.086366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:57.086574 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:57.086706 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:57.087025 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 07:07:57.087252 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":110,"server-negotiator.queue_time_us":225,"thread_start_us":127,"threads_started":1}
I20260812 07:07:57.092520 1160 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:38164:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 07:07:57.093397 1160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:07:57.099756 1319 tablet_service.cc:1467] Processing CreateTablet for tablet 89ca2ee4c933495a81a1118d7464c65d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b2b9e381e32b4c7c9550490e53580fc2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:07:57.099756 1481 tablet_service.cc:1467] Processing CreateTablet for tablet 89ca2ee4c933495a81a1118d7464c65d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b2b9e381e32b4c7c9550490e53580fc2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:07:57.099977 1637 tablet_service.cc:1467] Processing CreateTablet for tablet 89ca2ee4c933495a81a1118d7464c65d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b2b9e381e32b4c7c9550490e53580fc2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:07:57.100200 1481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ca2ee4c933495a81a1118d7464c65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:57.100200 1319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ca2ee4c933495a81a1118d7464c65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:57.100406 1637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ca2ee4c933495a81a1118d7464c65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:57.103293 1740 tablet_bootstrap.cc:492] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e: Bootstrap starting.
I20260812 07:07:57.103320 1739 tablet_bootstrap.cc:492] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e: Bootstrap starting.
I20260812 07:07:57.103351 1741 tablet_bootstrap.cc:492] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: Bootstrap starting.
I20260812 07:07:57.104221 1740 tablet_bootstrap.cc:654] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:57.104250 1739 tablet_bootstrap.cc:654] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:57.104254 1741 tablet_bootstrap.cc:654] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:57.105671 1740 tablet_bootstrap.cc:492] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e: No bootstrap required, opened a new log
I20260812 07:07:57.105768 1740 ts_tablet_manager.cc:1397] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:07:57.106108 1739 tablet_bootstrap.cc:492] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e: No bootstrap required, opened a new log
I20260812 07:07:57.106189 1741 tablet_bootstrap.cc:492] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: No bootstrap required, opened a new log
I20260812 07:07:57.106238 1739 ts_tablet_manager.cc:1397] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:07:57.106287 1741 ts_tablet_manager.cc:1397] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 07:07:57.106274 1740 raft_consensus.cc:348] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.106402 1740 raft_consensus.cc:374] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:57.106437 1740 raft_consensus.cc:729] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e290acb36ebe4464b39105dbed2f435e, State: Initialized, Role: FOLLOWER
I20260812 07:07:57.106560 1740 consensus_queue.cc:260] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.106829 1741 raft_consensus.cc:348] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.106953 1741 raft_consensus.cc:374] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:57.106988 1741 raft_consensus.cc:729] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee9bbfc5b254308832fc75e6b32a895, State: Initialized, Role: FOLLOWER
I20260812 07:07:57.106914 1746 tablet_replica.cc:439] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } }
I20260812 07:07:57.106945 1739 raft_consensus.cc:348] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.107064 1739 raft_consensus.cc:374] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:57.107089 1746 tablet_replica.cc:442] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:07:57.107097 1739 raft_consensus.cc:729] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda99d14cdea44a493b660c86ab3282e, State: Initialized, Role: FOLLOWER
I20260812 07:07:57.107103 1741 consensus_queue.cc:260] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.106909 1740 ts_tablet_manager.cc:1428] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:07:57.107398 1741 ts_tablet_manager.cc:1428] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:07:57.107355 1745 tablet_replica.cc:439] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } }
I20260812 07:07:57.107435 1739 consensus_queue.cc:260] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.107501 1745 tablet_replica.cc:442] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:07:57.107685 1750 tablet_replica.cc:439] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } }
I20260812 07:07:57.107822 1750 tablet_replica.cc:442] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:07:57.108047 1739 ts_tablet_manager.cc:1428] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 07:07:57.271027 1745 raft_consensus.cc:482] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:57.271248 1745 raft_consensus.cc:504] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.272908 1745 leader_election.cc:288] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bda99d14cdea44a493b660c86ab3282e (127.31.156.65:42487), e290acb36ebe4464b39105dbed2f435e (127.31.156.66:46721)
I20260812 07:07:57.278864 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.273259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:33923 (local address 127.31.156.65:42487)
0812 07:07:57.273547 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:57.273552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:57.274035 (+ 483us) server_negotiation.cc:407] Connection header received
0812 07:07:57.274111 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:57.274114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:57.274174 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:57.274274 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:57.274644 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.275541 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.277363 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.278074 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.278226 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.278290 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:57.278567 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 07:07:57.278697 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":87,"threads_started":1}
I20260812 07:07:57.278877 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.273112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:42487 (local address 127.31.156.67:33923)
0812 07:07:57.273777 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:57.273796 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:57.273927 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:57.274299 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:57.274305 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:57.274327 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:57.274485 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.274494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.275704 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.275708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.277165 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.277179 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.278236 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.278244 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.278369 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.278434 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 07:07:57.278540 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 07:07:57.278644 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":5,"thread_start_us":81,"threads_started":1}
I20260812 07:07:57.279495 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.274129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:33035 (local address 127.31.156.66:46721)
0812 07:07:57.274493 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:57.274499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:57.274514 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:07:57.274572 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:57.274578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:57.274636 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:57.274751 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:57.275115 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.276160 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.277661 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.278173 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.278308 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.278377 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:57.278938 (+ 561us) server_negotiation.cc:299] Negotiation successful
0812 07:07:57.279351 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":80,"threads_started":1}
I20260812 07:07:57.279485 1343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ca2ee4c933495a81a1118d7464c65d" candidate_uuid: "4ee9bbfc5b254308832fc75e6b32a895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda99d14cdea44a493b660c86ab3282e" is_pre_election: true
I20260812 07:07:57.279749 1786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.273703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:46721 (local address 127.31.156.67:33035)
0812 07:07:57.273958 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:57.273974 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:57.274083 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:57.274760 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:57.274763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:57.274782 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:57.274963 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.274972 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.276315 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.276319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.277523 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.277532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.278384 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.278389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.278563 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.278592 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:07:57.278694 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 07:07:57.278794 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":29,"thread_start_us":90,"threads_started":1}
I20260812 07:07:57.279796 1343 raft_consensus.cc:2454] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ee9bbfc5b254308832fc75e6b32a895 in term 0.
I20260812 07:07:57.280298 1503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ca2ee4c933495a81a1118d7464c65d" candidate_uuid: "4ee9bbfc5b254308832fc75e6b32a895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e290acb36ebe4464b39105dbed2f435e" is_pre_election: true
I20260812 07:07:57.280468 1503 raft_consensus.cc:2454] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ee9bbfc5b254308832fc75e6b32a895 in term 0.
I20260812 07:07:57.280335 1586 leader_election.cc:302] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee9bbfc5b254308832fc75e6b32a895, bda99d14cdea44a493b660c86ab3282e; no voters:
I20260812 07:07:57.280836 1745 raft_consensus.cc:2781] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:07:57.280913 1745 raft_consensus.cc:482] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:07:57.280953 1745 raft_consensus.cc:3037] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:57.282471 1745 raft_consensus.cc:504] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.282980 1745 leader_election.cc:288] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [CANDIDATE]: Term 1 election: Requested vote from peers bda99d14cdea44a493b660c86ab3282e (127.31.156.65:42487), e290acb36ebe4464b39105dbed2f435e (127.31.156.66:46721)
I20260812 07:07:57.283476 1343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ca2ee4c933495a81a1118d7464c65d" candidate_uuid: "4ee9bbfc5b254308832fc75e6b32a895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda99d14cdea44a493b660c86ab3282e"
I20260812 07:07:57.283659 1343 raft_consensus.cc:3037] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:57.283628 1503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ca2ee4c933495a81a1118d7464c65d" candidate_uuid: "4ee9bbfc5b254308832fc75e6b32a895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e290acb36ebe4464b39105dbed2f435e"
I20260812 07:07:57.283758 1503 raft_consensus.cc:3037] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:57.285100 1343 raft_consensus.cc:2454] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ee9bbfc5b254308832fc75e6b32a895 in term 1.
I20260812 07:07:57.285100 1503 raft_consensus.cc:2454] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ee9bbfc5b254308832fc75e6b32a895 in term 1.
I20260812 07:07:57.285498 1586 leader_election.cc:302] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee9bbfc5b254308832fc75e6b32a895, bda99d14cdea44a493b660c86ab3282e; no voters:
I20260812 07:07:57.285740 1745 raft_consensus.cc:2781] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:57.285954 1745 raft_consensus.cc:686] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [term 1 LEADER]: Becoming Leader. State: Replica: 4ee9bbfc5b254308832fc75e6b32a895, State: Running, Role: LEADER
I20260812 07:07:57.286244 1745 consensus_queue.cc:237] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } }
I20260812 07:07:57.287946 1788 tablet_replica.cc:439] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: TxnStatusTablet state changed. Reason: New leader 4ee9bbfc5b254308832fc75e6b32a895. Latest consensus state: current_term: 1 leader_uuid: "4ee9bbfc5b254308832fc75e6b32a895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } }
I20260812 07:07:57.288103 1788 tablet_replica.cc:442] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:07:57.289399 1163 catalog_manager.cc:5495] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ee9bbfc5b254308832fc75e6b32a895 (127.31.156.67). New cstate: current_term: 1 leader_uuid: "4ee9bbfc5b254308832fc75e6b32a895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } health_report { overall_health: HEALTHY } } }
I20260812 07:07:57.293105 1790 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 07:07:57.293334 1790 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 07:07:57.304924 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.298688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.31.156.67:40093)
0812 07:07:57.298991 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:57.298999 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:57.299013 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:07:57.299092 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:57.299095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:57.299191 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:57.299306 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:07:57.299641 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.301361 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.302868 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.303179 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.303394 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.303812 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:07:57.304424 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:07:57.304527 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:57.304685 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 07:07:57.304757 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":131,"threads_started":1}
I20260812 07:07:57.304935 1779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.298497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:40093 (local address 127.0.0.1:53502)
0812 07:07:57.298736 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:57.298754 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:57.298876 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:57.299367 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:57.299371 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:57.299380 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:07:57.299504 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.299511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.301554 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.301558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.302644 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.302701 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.303422 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.303426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.303540 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.303615 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:07:57.304543 (+ 928us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:07:57.304566 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:07:57.304653 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 07:07:57.304773 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 07:07:57.308745 1503 raft_consensus.cc:1261] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e [term 1 FOLLOWER]: Refusing update from remote peer 4ee9bbfc5b254308832fc75e6b32a895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:57.309034 1343 raft_consensus.cc:1261] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e [term 1 FOLLOWER]: Refusing update from remote peer 4ee9bbfc5b254308832fc75e6b32a895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:57.309330 1788 consensus_queue.cc:1035] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:57.309653 1745 consensus_queue.cc:1035] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:57.311543 1746 tablet_replica.cc:439] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e: TxnStatusTablet state changed. Reason: New leader 4ee9bbfc5b254308832fc75e6b32a895. Latest consensus state: current_term: 1 leader_uuid: "4ee9bbfc5b254308832fc75e6b32a895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } }
I20260812 07:07:57.311681 1746 tablet_replica.cc:442] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:07:57.311764 1750 tablet_replica.cc:439] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e: TxnStatusTablet state changed. Reason: New leader 4ee9bbfc5b254308832fc75e6b32a895. Latest consensus state: current_term: 1 leader_uuid: "4ee9bbfc5b254308832fc75e6b32a895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } }
I20260812 07:07:57.311911 1750 tablet_replica.cc:442] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:07:57.314733 1788 tablet_replica.cc:439] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ee9bbfc5b254308832fc75e6b32a895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } }
I20260812 07:07:57.314886 1788 tablet_replica.cc:442] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:07:57.316280 1750 tablet_replica.cc:439] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ee9bbfc5b254308832fc75e6b32a895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } }
I20260812 07:07:57.316419 1750 tablet_replica.cc:442] T 89ca2ee4c933495a81a1118d7464c65d P bda99d14cdea44a493b660c86ab3282e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:07:57.316998 1788 tablet_replica.cc:439] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ee9bbfc5b254308832fc75e6b32a895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } }
I20260812 07:07:57.317138 1788 tablet_replica.cc:442] T 89ca2ee4c933495a81a1118d7464c65d P 4ee9bbfc5b254308832fc75e6b32a895: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:07:57.318282 1746 tablet_replica.cc:439] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ee9bbfc5b254308832fc75e6b32a895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda99d14cdea44a493b660c86ab3282e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42487 } } peers { permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 } } peers { permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 } } }
I20260812 07:07:57.318406 1746 tablet_replica.cc:442] T 89ca2ee4c933495a81a1118d7464c65d P e290acb36ebe4464b39105dbed2f435e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:07:57.334932 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.330246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:39241 (local address 127.31.156.67:40093)
0812 07:07:57.330402 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:57.330408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:57.330986 (+ 578us) server_negotiation.cc:407] Connection header received
0812 07:07:57.331131 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:57.331134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:57.331187 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:57.331349 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:07:57.331628 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.332315 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.333656 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.333923 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.334031 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.334398 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:07:57.334475 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:07:57.334536 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:57.334752 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 07:07:57.334814 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:07:57.334947 1805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.330177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:40093 (local address 127.31.156.65:39241)
0812 07:07:57.330879 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:57.330895 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:57.331015 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:57.331377 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:57.331380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:57.331389 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:07:57.331510 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.331516 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.332447 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.332451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.333470 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.333482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.334062 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.334066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.334169 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.334239 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:07:57.334565 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:07:57.334572 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:07:57.334641 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 07:07:57.334767 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":134,"threads_started":1}
I20260812 07:07:57.343236 1663 raft_consensus.cc:1261] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895 [term 1 FOLLOWER]: Refusing update from remote peer bda99d14cdea44a493b660c86ab3282e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:57.343312 1503 raft_consensus.cc:1261] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e [term 1 FOLLOWER]: Refusing update from remote peer bda99d14cdea44a493b660c86ab3282e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:57.343741 1750 consensus_queue.cc:1035] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e290acb36ebe4464b39105dbed2f435e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:57.343952 1747 consensus_queue.cc:1035] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee9bbfc5b254308832fc75e6b32a895" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:57.372045 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.367941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:56593 (local address 127.31.156.65:42487)
0812 07:07:57.368108 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:57.368115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:57.368406 (+ 291us) server_negotiation.cc:407] Connection header received
0812 07:07:57.368622 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:57.368625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:57.368679 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:57.368762 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:07:57.369052 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.369818 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.370911 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.371145 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.371259 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.371563 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:07:57.371649 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:07:57.371738 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:57.371827 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 07:07:57.371882 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 07:07:57.372093 1813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.367851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:42487 (local address 127.31.156.67:56593)
0812 07:07:57.368304 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:57.368317 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:57.368503 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:57.368783 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:57.368786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:57.368793 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:07:57.368942 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.368948 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.369913 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.369919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.370791 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:57.370801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.371264 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.371268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:57.371368 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.371423 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:07:57.371742 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:07:57.371751 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 07:07:57.371818 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 07:07:57.371901 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":140,"threads_started":1}
I20260812 07:07:57.399945 1158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:57.400203 1158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:57.402678 1158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f2d6092ae364aeea6ad6b4b0205a7ae: Sending DeleteTablet for 3 replicas of tablet 0205cb91b8ee43caaac08d953cc6e1e1
I20260812 07:07:57.403328 1319 tablet_service.cc:1514] Processing DeleteTablet for tablet 0205cb91b8ee43caaac08d953cc6e1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:57 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:43772
I20260812 07:07:57.403570 1481 tablet_service.cc:1514] Processing DeleteTablet for tablet 0205cb91b8ee43caaac08d953cc6e1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:57 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:44778
I20260812 07:07:57.403904 1819 tablet_replica.cc:330] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e: stopping tablet replica
I20260812 07:07:57.403955 1820 tablet_replica.cc:330] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e: stopping tablet replica
I20260812 07:07:57.404107 1820 raft_consensus.cc:2229] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:57.404136 1819 raft_consensus.cc:2229] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:07:57.404250 1820 raft_consensus.cc:2258] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:57.404482 1819 raft_consensus.cc:2258] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:57.405282 1819 ts_tablet_manager.cc:1905] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:57.405464 1637 tablet_service.cc:1514] Processing DeleteTablet for tablet 0205cb91b8ee43caaac08d953cc6e1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:57 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:53492
I20260812 07:07:57.405757 1820 ts_tablet_manager.cc:1905] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:57.406736 1822 tablet_replica.cc:330] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895: stopping tablet replica
I20260812 07:07:57.406890 1822 raft_consensus.cc:2229] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:57.407027 1822 raft_consensus.cc:2258] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:57.407783 1822 ts_tablet_manager.cc:1905] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:57.408473 1819 ts_tablet_manager.cc:1918] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:07:57.408561 1819 log.cc:1196] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-0/wal/wals/0205cb91b8ee43caaac08d953cc6e1e1
I20260812 07:07:57.408875 1819 ts_tablet_manager.cc:1939] T 0205cb91b8ee43caaac08d953cc6e1e1 P bda99d14cdea44a493b660c86ab3282e: Deleting consensus metadata
I20260812 07:07:57.409538 1820 ts_tablet_manager.cc:1918] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:07:57.409708 1820 log.cc:1196] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/0205cb91b8ee43caaac08d953cc6e1e1
I20260812 07:07:57.410084 1820 ts_tablet_manager.cc:1939] T 0205cb91b8ee43caaac08d953cc6e1e1 P e290acb36ebe4464b39105dbed2f435e: Deleting consensus metadata
I20260812 07:07:57.410181 1150 catalog_manager.cc:4841] TS bda99d14cdea44a493b660c86ab3282e (127.31.156.65:42487): tablet 0205cb91b8ee43caaac08d953cc6e1e1 (table test-table [id=c03743d73cd94ace9d9430543c00776a]) successfully deleted
I20260812 07:07:57.411089 1145 catalog_manager.cc:4841] TS e290acb36ebe4464b39105dbed2f435e (127.31.156.66:46721): tablet 0205cb91b8ee43caaac08d953cc6e1e1 (table test-table [id=c03743d73cd94ace9d9430543c00776a]) successfully deleted
I20260812 07:07:57.411309 1822 ts_tablet_manager.cc:1918] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:07:57.411398 1822 log.cc:1196] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/0205cb91b8ee43caaac08d953cc6e1e1
I20260812 07:07:57.411682 1822 ts_tablet_manager.cc:1939] T 0205cb91b8ee43caaac08d953cc6e1e1 P 4ee9bbfc5b254308832fc75e6b32a895: Deleting consensus metadata
I20260812 07:07:57.412392 1145 catalog_manager.cc:4841] TS 4ee9bbfc5b254308832fc75e6b32a895 (127.31.156.67:40093): tablet 0205cb91b8ee43caaac08d953cc6e1e1 (table test-table [id=c03743d73cd94ace9d9430543c00776a]) successfully deleted
Aug 12 07:07:57 dist-test-slave-csg5 krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518476, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:07:57.421970 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.406082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35760 (local address 127.31.156.65:42487)
0812 07:07:57.406204 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:57.406209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:57.411019 (+ 4810us) server_negotiation.cc:407] Connection header received
0812 07:07:57.411217 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:57.411220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:57.411275 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:57.411327 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:57.412450 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.413182 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.414467 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.414757 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.414850 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.418029 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:57.418043 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:57.418056 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:57.418083 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:57.419972 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:57.420482 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:57.420488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:57.420489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:57.420576 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:57.420945 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:57.420949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:57.420951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:57.421235 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:57.421462 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:57.421655 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 07:07:57.421797 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 07:07:57.422461 1389 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35760
I20260812 07:07:57.432775 1780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.424556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.31.156.126:44441)
0812 07:07:57.424716 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:57.424721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:57.424737 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:07:57.424828 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:57.424831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:57.424878 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:57.424975 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:57.425647 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.426194 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.427088 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.427427 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.427542 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.429233 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:57.429236 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:57.429238 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:57.429267 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:57.430943 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:57.431419 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:57.431423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:57.431427 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:57.431517 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:57.431814 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:57.431818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:57.431820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:57.432022 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:57.432182 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:57.432447 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 07:07:57.432594 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 07:07:57.433290 1158 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56998
I20260812 07:07:57.434542 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 1247
I20260812 07:07:57.452087 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 1406
I20260812 07:07:57.462520 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 1567
I20260812 07:07:57.473929 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 1128
2026-08-12T07:07:57Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3358 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:57 dist-test-slave-csg5 krb5kdc[1829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:57 dist-test-slave-csg5 krb5kdc[1829](info): set up 2 sockets
Aug 12 07:07:57 dist-test-slave-csg5 krb5kdc[1829](info): commencing operation
krb5kdc: starting...
W20260812 07:07:58.666630 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:07:58 dist-test-slave-csg5 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:07:58.820098 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:41753
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:41753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:58.939848 1843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:58.940176 1843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:58.940217 1843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:58.944340 1843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:58.944416 1843 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 07:07:58.944434 1843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:58.944454 1843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:58.944471 1843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:58.944487 1843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:58.950089 1843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:41753
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:41753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:58.951370 1843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:58.952453 1843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:58.961866 1848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:58.962028 1853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:58.962101 1849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:58.962466 1843 server_base.cc:1034] running on GCE node
I20260812 07:07:58.963197 1843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:58.964521 1843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:58.965706 1843 hybrid_clock.cc:648] HybridClock initialized: now 1786518478965652 us; error 70 us; skew 500 ppm
Aug 12 07:07:58 dist-test-slave-csg5 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518478, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:58.973219 1843 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:07:58.974390 1843 webserver.cc:460] Webserver started at http://127.31.156.126:46261/ using document root <none> and password file <none>
I20260812 07:07:58.975049 1843 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:58.975127 1843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:58.975315 1843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:58.977502 1843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "99ff1ff718fd44d5a854dae619519b76"
format_stamp: "Formatted at 2026-08-12 07:07:58 on dist-test-slave-csg5"
server_key: "5a481c03e7eb175987ec9d7f7ab3c352"
server_key_iv: "748699b52e992d8e6fba2dc0a6a6322a"
server_key_version: "encryptionkey@0"
I20260812 07:07:58.978168 1843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "99ff1ff718fd44d5a854dae619519b76"
format_stamp: "Formatted at 2026-08-12 07:07:58 on dist-test-slave-csg5"
server_key: "5a481c03e7eb175987ec9d7f7ab3c352"
server_key_iv: "748699b52e992d8e6fba2dc0a6a6322a"
server_key_version: "encryptionkey@0"
I20260812 07:07:58.982546 1843 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:07:58.985474 1860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:58.987028 1843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:07:58.987188 1843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "99ff1ff718fd44d5a854dae619519b76"
format_stamp: "Formatted at 2026-08-12 07:07:58 on dist-test-slave-csg5"
server_key: "5a481c03e7eb175987ec9d7f7ab3c352"
server_key_iv: "748699b52e992d8e6fba2dc0a6a6322a"
server_key_version: "encryptionkey@0"
I20260812 07:07:58.987331 1843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:59.024850 1843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:59.028904 1843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:59.029201 1843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:59.038880 1843 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:41753
I20260812 07:07:59.038909 1927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:41753 every 8 connection(s)
I20260812 07:07:59.039974 1843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:59.043254 1928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:59.049703 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 1843
I20260812 07:07:59.049850 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:07:59.050050 1928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76: Bootstrap starting.
I20260812 07:07:59.050165 32369 external_mini_cluster.cc:1428] Setting key 70623629cdc13d73adc6b7555099e978
I20260812 07:07:59.053244 1928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:59.054137 1928 log.cc:824] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:59.056597 1928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76: No bootstrap required, opened a new log
I20260812 07:07:59.060884 1928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ff1ff718fd44d5a854dae619519b76" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41753 } }
I20260812 07:07:59.061280 1928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:59.061375 1928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ff1ff718fd44d5a854dae619519b76, State: Initialized, Role: FOLLOWER
I20260812 07:07:59.061898 1928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ff1ff718fd44d5a854dae619519b76" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41753 } }
I20260812 07:07:59.062078 1928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 12 07:07:59 dist-test-slave-csg5 krb5kdc[1829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:59.062183 1928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:59.062316 1928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:59.063565 1928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ff1ff718fd44d5a854dae619519b76" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41753 } }
I20260812 07:07:59.063989 1928 leader_election.cc:302] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99ff1ff718fd44d5a854dae619519b76; no voters:
I20260812 07:07:59.064371 1928 leader_election.cc:288] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:59.064543 1933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:59.064824 1933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [term 1 LEADER]: Becoming Leader. State: Replica: 99ff1ff718fd44d5a854dae619519b76, State: Running, Role: LEADER
I20260812 07:07:59.065304 1933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ff1ff718fd44d5a854dae619519b76" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41753 } }
I20260812 07:07:59.066068 1928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:59.067387 1935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99ff1ff718fd44d5a854dae619519b76. Latest consensus state: current_term: 1 leader_uuid: "99ff1ff718fd44d5a854dae619519b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ff1ff718fd44d5a854dae619519b76" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41753 } } }
I20260812 07:07:59.067916 1935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:59.067768 1936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99ff1ff718fd44d5a854dae619519b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ff1ff718fd44d5a854dae619519b76" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41753 } } }
I20260812 07:07:59.068300 1936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:59.070562 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.051961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.31.156.126:41753)
0812 07:07:59.052520 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.052538 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.052748 (+ 210us) server_negotiation.cc:407] Connection header received
0812 07:07:59.053665 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.053685 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.054036 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.054432 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:59.055409 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.056597 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.057965 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.059265 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.059416 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.062968 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:59.062974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:59.062990 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:59.063025 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:59.065660 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.066338 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.066345 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.066351 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.066525 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.066954 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.066962 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.066967 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.067653 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:59.067918 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.069126 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.069417 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":178,"threads_started":1}
W20260812 07:07:59.072669 1950 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:07:59.072813 1950 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:07:59.072921 1947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:59.073990 1947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:59.080139 1947 catalog_manager.cc:1347] Generated new cluster ID: f7efd440028c440db8590942be093aa2
I20260812 07:07:59.080237 1947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:59.095060 1947 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:59.096464 1947 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:59.106495 1947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76: Generated new TSK 0
I20260812 07:07:59.107463 1947 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:07:59.151827 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:41753
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:59.277834 1957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:59.278226 1957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:59.278318 1957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:59.282740 1957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:59.282864 1957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:59.282948 1957 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 07:07:59.282994 1957 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:07:59.283070 1957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:07:59.289266 1957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.156.126:41753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:59.290566 1957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:59.291836 1957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:59.303516 1964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:59.303617 1957 server_base.cc:1034] running on GCE node
W20260812 07:07:59.303521 1967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:59.303520 1965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:59.304415 1957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:59.305300 1957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:59.306532 1957 hybrid_clock.cc:648] HybridClock initialized: now 1786518479306477 us; error 75 us; skew 500 ppm
Aug 12 07:07:59 dist-test-slave-csg5 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518479, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:59.310127 1957 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:07:59.311354 1957 webserver.cc:460] Webserver started at http://127.31.156.65:45635/ using document root <none> and password file <none>
I20260812 07:07:59.311995 1957 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:59.312095 1957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:59.312371 1957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:59.314592 1957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "c2302f3668514346910c50cadda3e7f5"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-csg5"
server_key: "ec7b9cdb3f23de0c515a63c07c9627fb"
server_key_iv: "e75adb05858db42af25daaf589e26c84"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.315609 1957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "c2302f3668514346910c50cadda3e7f5"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-csg5"
server_key: "ec7b9cdb3f23de0c515a63c07c9627fb"
server_key_iv: "e75adb05858db42af25daaf589e26c84"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.321261 1957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:07:59.324219 1975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:59.325690 1957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:07:59.325917 1957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "c2302f3668514346910c50cadda3e7f5"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-csg5"
server_key: "ec7b9cdb3f23de0c515a63c07c9627fb"
server_key_iv: "e75adb05858db42af25daaf589e26c84"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.326067 1957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:59.344725 1957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:59.349190 1957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:59.349434 1957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:59.351279 1957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:59.351361 1957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:59.351438 1957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:59.351481 1957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:07:59 dist-test-slave-csg5 krb5kdc[1829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518479, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:59.363092 1957 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:34199
I20260812 07:07:59.363216 2109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:34199 every 8 connection(s)
I20260812 07:07:59.364382 1957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:07:59.367165 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.352091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:47343 (local address 127.31.156.126:41753)
0812 07:07:59.352271 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.352279 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.353688 (+ 1409us) server_negotiation.cc:407] Connection header received
0812 07:07:59.354975 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.354979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.355045 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.355134 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:59.356604 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.357267 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.358874 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.359162 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.359297 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.362604 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:59.362608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:59.362611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:59.362684 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:59.364546 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.365266 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.365270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.365271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.365379 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.365752 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.365754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.365755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.365994 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:59.366077 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.366815 (+ 738us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.367005 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:07:59.368000 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.353014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41753 (local address 127.31.156.65:47343)
0812 07:07:59.353510 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:59.353563 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:59.354742 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:59.355349 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:59.355356 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:59.355846 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:59.356406 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.356431 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.357413 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.357417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.358719 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.358731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.359358 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.359362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.359604 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.360378 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:59.360398 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:59.362396 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:59.364678 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.364684 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.364697 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.365124 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.365491 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.365494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.365496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.365661 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.366077 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:59.366082 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:59.366479 (+ 397us) client_negotiation.cc:769] Sending connection context
0812 07:07:59.366705 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 07:07:59.367018 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":144,"threads_started":1}
I20260812 07:07:59.369931 2110 heartbeater.cc:347] Connected to a master server at 127.31.156.126:41753
I20260812 07:07:59.370137 2110 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:59.370792 2110 heartbeater.cc:511] Master 127.31.156.126:41753 requested a full tablet report, sending...
I20260812 07:07:59.371224 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 1957
I20260812 07:07:59.371366 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:07:59.371685 32369 external_mini_cluster.cc:1428] Setting key c651b6f11509f4267b7049ea56bc0dd1
I20260812 07:07:59.373270 1877 ts_manager.cc:194] Registered new tserver with Master: c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199)
I20260812 07:07:59.374493 1877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:47343
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:07:59.383916 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.375222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:58477 (local address 127.31.156.126:41753)
0812 07:07:59.375354 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.375360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.375377 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:07:59.375431 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.375435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.375488 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.375575 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:59.376363 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.377127 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.378513 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.378862 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.378961 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.380462 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:59.380465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:59.380467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:59.380492 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:59.381986 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.382546 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.382548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.382550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.382632 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.383007 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.383009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.383010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.383214 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:59.383378 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.383672 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.383784 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 07:07:59.383958 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.372574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41753 (local address 127.31.156.65:58477)
0812 07:07:59.372888 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:59.372901 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:59.372985 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:59.375722 (+ 2737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:59.375728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:59.376025 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:59.376203 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.376214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.377301 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.377308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.378360 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.378379 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.379007 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.379014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.379132 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.379712 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:59.379734 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:59.380281 (+ 547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:59.382115 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.382121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.382125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.382427 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.382772 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.382777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.382783 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.382904 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.383348 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:59.383354 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:59.383481 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 07:07:59.383609 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 07:07:59.383782 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:07:59.408244 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:41753
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:59.539273 2122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:59.539615 2122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:59.539678 2122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:59.543994 2122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:59.544078 2122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:59.544122 2122 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 07:07:59.544137 2122 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:07:59.544181 2122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:07:59.549969 2122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.156.126:41753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:59.551271 2122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:59.552377 2122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:59.563091 2133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:59.563189 2128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:59.563210 2129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:59.563491 2122 server_base.cc:1034] running on GCE node
I20260812 07:07:59.564060 2122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:59.564895 2122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:59.566102 2122 hybrid_clock.cc:648] HybridClock initialized: now 1786518479566046 us; error 67 us; skew 500 ppm
Aug 12 07:07:59 dist-test-slave-csg5 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518479, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:59.569990 2122 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:07:59.571295 2122 webserver.cc:460] Webserver started at http://127.31.156.66:36341/ using document root <none> and password file <none>
I20260812 07:07:59.571947 2122 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:59.572052 2122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:59.572316 2122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:59.574572 2122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "96eb3581fba04a3da45b174be1fa4940"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-csg5"
server_key: "cea62cd2d42663296b80ae3a44966679"
server_key_iv: "41ec749361f710f0c05372d93dafac17"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.575307 2122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "96eb3581fba04a3da45b174be1fa4940"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-csg5"
server_key: "cea62cd2d42663296b80ae3a44966679"
server_key_iv: "41ec749361f710f0c05372d93dafac17"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.579707 2122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:07:59.582782 2141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:59.584504 2122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:07:59.584718 2122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "96eb3581fba04a3da45b174be1fa4940"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-csg5"
server_key: "cea62cd2d42663296b80ae3a44966679"
server_key_iv: "41ec749361f710f0c05372d93dafac17"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.584894 2122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:59.624636 2122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:59.628623 2122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:59.628883 2122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:59.630766 2122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:59.630848 2122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:59.630939 2122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:59.630986 2122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:07:59 dist-test-slave-csg5 krb5kdc[1829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518479, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:59.643860 2122 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:43355
I20260812 07:07:59.643895 2274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:43355 every 8 connection(s)
I20260812 07:07:59.645350 2122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:07:59.646991 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.631903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:43921 (local address 127.31.156.126:41753)
0812 07:07:59.632101 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.632107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.633368 (+ 1261us) server_negotiation.cc:407] Connection header received
0812 07:07:59.634971 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.634977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.635047 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.635171 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:59.636779 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.637389 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.639061 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.639328 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.639460 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.642447 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:59.642452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:59.642455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:59.642490 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:59.644449 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.645036 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.645040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.645042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.645162 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.645542 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.645545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.645547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.645849 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:59.645962 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.646728 (+ 766us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.646854 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:07:59.647950 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.632585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41753 (local address 127.31.156.66:43921)
0812 07:07:59.633158 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:59.633226 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:59.634585 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:59.635390 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:59.635397 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:59.635929 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:59.636594 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.636610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.637543 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.637547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.638896 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.638910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.639539 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.639545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.639753 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.640499 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:59.640529 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:59.642233 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:59.644588 (+ 2355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.644594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.644608 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.644914 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.645271 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.645274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.645277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.645421 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.646023 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:59.646033 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:59.646385 (+ 352us) client_negotiation.cc:769] Sending connection context
0812 07:07:59.646591 (+ 206us) client_negotiation.cc:240] Negotiation successful
0812 07:07:59.647012 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":112,"threads_started":1}
I20260812 07:07:59.648236 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 2122
I20260812 07:07:59.648357 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:07:59.648659 32369 external_mini_cluster.cc:1428] Setting key e48c06f8fe0c490341aa84106ebc4c53
I20260812 07:07:59.650244 2275 heartbeater.cc:347] Connected to a master server at 127.31.156.126:41753
I20260812 07:07:59.650449 2275 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:59.651196 2275 heartbeater.cc:511] Master 127.31.156.126:41753 requested a full tablet report, sending...
I20260812 07:07:59.652518 1874 ts_manager.cc:194] Registered new tserver with Master: 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355)
I20260812 07:07:59.653492 1874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:43921
I20260812 07:07:59.663427 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.653156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41753 (local address 127.31.156.66:34919)
0812 07:07:59.653487 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:59.653501 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:59.653635 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:59.653919 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:59.653925 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:59.654216 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:59.654392 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.654403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.655574 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.655577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.656420 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.656432 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.657150 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.657153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.657322 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.658051 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:59.658064 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:59.658452 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:59.661179 (+ 2727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.661183 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.661184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.661546 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.661986 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.661989 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.661990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.662104 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.662809 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:59.662812 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:59.662922 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 07:07:59.663055 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 07:07:59.663210 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":3,"thread_start_us":143,"threads_started":1}
I20260812 07:07:59.663609 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.652940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:34919 (local address 127.31.156.126:41753)
0812 07:07:59.653108 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.653116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.653582 (+ 466us) server_negotiation.cc:407] Connection header received
0812 07:07:59.653740 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.653743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.653791 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.653881 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:59.654556 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.655411 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.656568 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.657008 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.657161 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.658632 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:59.658639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:59.658643 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:59.658715 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:59.660971 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.661686 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.661693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.661696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.661832 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.662226 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.662231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.662235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.662626 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:59.662802 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.663177 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.663345 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:07:59.692181 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:41753
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:59.819973 2285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:59.820415 2285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:59.820513 2285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:59.824899 2285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:59.825004 2285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:59.825081 2285 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 07:07:59.825130 2285 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:07:59.825258 2285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:07:59.831271 2285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.156.126:41753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:07:59.832480 2285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:59.833673 2285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:59.844834 2292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:59.845053 2285 server_base.cc:1034] running on GCE node
W20260812 07:07:59.844973 2294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:59.844857 2291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:59.845741 2285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:59.846616 2285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:59.847903 2285 hybrid_clock.cc:648] HybridClock initialized: now 1786518479847879 us; error 36 us; skew 500 ppm
Aug 12 07:07:59 dist-test-slave-csg5 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518479, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:59.851616 2285 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:07:59.852735 2285 webserver.cc:460] Webserver started at http://127.31.156.67:44753/ using document root <none> and password file <none>
I20260812 07:07:59.853395 2285 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:59.853482 2285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:59.853686 2285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:59.856281 2285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "3d9f5e32109f478f8493792225db338b"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-csg5"
server_key: "d8a820d13df62c76d8fdb94a2e3c2765"
server_key_iv: "37c21b9edd77fa5b7d3cb67d46e28e48"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.856948 2285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "3d9f5e32109f478f8493792225db338b"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-csg5"
server_key: "d8a820d13df62c76d8fdb94a2e3c2765"
server_key_iv: "37c21b9edd77fa5b7d3cb67d46e28e48"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.861845 2285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:07:59.864939 2302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:59.867010 2285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:07:59.867175 2285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "3d9f5e32109f478f8493792225db338b"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-csg5"
server_key: "d8a820d13df62c76d8fdb94a2e3c2765"
server_key_iv: "37c21b9edd77fa5b7d3cb67d46e28e48"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.867339 2285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:59.881733 2285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:59.886051 2285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:59.886339 2285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:59.888365 2285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:59.888458 2285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:59.888533 2285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:59.888582 2285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:07:59 dist-test-slave-csg5 krb5kdc[1829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518479, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:59.901559 2285 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:35485
I20260812 07:07:59.901665 2431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:35485 every 8 connection(s)
I20260812 07:07:59.902709 2285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:07:59.906215 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.889795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:45343 (local address 127.31.156.126:41753)
0812 07:07:59.889980 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.889985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.891296 (+ 1311us) server_negotiation.cc:407] Connection header received
0812 07:07:59.892694 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.892700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.892778 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.892905 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:59.894328 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.895361 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.897421 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.897881 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.898042 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.901747 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:59.901754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:59.901759 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:59.901803 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:59.903949 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.904532 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.904536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.904537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.904631 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.905007 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.905011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.905012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.905246 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:59.905362 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.905887 (+ 525us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.906026 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:07:59.907172 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.890346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41753 (local address 127.31.156.67:45343)
0812 07:07:59.891119 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:59.891170 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:59.892379 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:59.893124 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:59.893131 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:59.893543 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:59.894168 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.894183 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.895615 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.895619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.897229 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.897246 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.898119 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.898125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.898383 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.899147 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:59.899172 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:59.901510 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:59.904091 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.904098 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.904114 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.904400 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.904743 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.904746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.904748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.904902 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.905361 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:59.905368 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:59.905636 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 07:07:59.905842 (+ 206us) client_negotiation.cc:240] Negotiation successful
0812 07:07:59.906179 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":201,"threads_started":1}
I20260812 07:07:59.908648 2432 heartbeater.cc:347] Connected to a master server at 127.31.156.126:41753
I20260812 07:07:59.908963 2432 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:59.909651 2432 heartbeater.cc:511] Master 127.31.156.126:41753 requested a full tablet report, sending...
I20260812 07:07:59.911116 1877 ts_manager.cc:194] Registered new tserver with Master: 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485)
I20260812 07:07:59.912035 1877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:45343
I20260812 07:07:59.912073 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 2285
I20260812 07:07:59.912213 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:07:59.912572 32369 external_mini_cluster.cc:1428] Setting key f2820afb17dc065cf2d7936004160d4f
I20260812 07:07:59.915652 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:07:59.920763 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.911616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:58789 (local address 127.31.156.126:41753)
0812 07:07:59.911787 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.911792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.912768 (+ 976us) server_negotiation.cc:407] Connection header received
0812 07:07:59.912903 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.912908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.912965 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.913057 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:59.913740 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.914481 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.915586 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.915894 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.916016 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.917276 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:59.917280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:59.917283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:59.917309 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:59.918901 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.919379 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.919385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.919387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.919475 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.919809 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.919813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.919815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.920041 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:59.920143 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.920502 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.920620 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 07:07:59.920946 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.911483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41753 (local address 127.31.156.67:58789)
0812 07:07:59.912657 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:59.912675 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:59.912797 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:59.913083 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:59.913089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:59.913391 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:59.913607 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.913617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.914680 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.914684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.915455 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.915466 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.916022 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.916026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.916120 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.916715 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:59.916728 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:59.917115 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:59.919022 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.919025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.919028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.919261 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.919578 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:59.919584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:59.919589 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:59.919702 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:59.920169 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:59.920175 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:59.920305 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 07:07:59.920429 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 07:07:59.920618 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"mutex_wait_us":2,"thread_start_us":201,"threads_started":1}
Aug 12 07:07:59 dist-test-slave-csg5 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:07:59 dist-test-slave-csg5 krb5kdc[1829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:07:59.948796 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.938749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.31.156.126:41753)
0812 07:07:59.938917 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.938921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.938937 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:07:59.938983 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.938985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.939030 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.939136 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:59.940024 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.940625 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.941625 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.941880 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.942010 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.944966 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:59.944969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:59.944972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:59.945003 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:59.946810 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.947330 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.947334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.947336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.947438 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:59.947765 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:59.947768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:59.947770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:59.947993 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:59.948151 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.948430 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.948578 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:07:59.952888 1877 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:07:59.956501 1877 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:07:59.975031 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.967208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35485 (local address 127.0.0.1:52440)
0812 07:07:59.968123 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:59.968218 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:59.968719 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:59.969453 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:59.969461 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:59.969497 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:59.969795 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.969804 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.971507 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.971511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.973166 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.973184 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.974232 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.974236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.974384 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.974437 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 07:07:59.974579 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 07:07:59.974835 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":266,"spinlock_wait_cycles":741504,"thread_start_us":135,"threads_started":1}
I20260812 07:07:59.975296 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.967585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.31.156.65:34199)
0812 07:07:59.968572 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.968587 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.968621 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:07:59.969071 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.969078 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.969311 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.969449 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:59.969971 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.971065 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.973305 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.974236 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.974339 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.974501 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.975009 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.975133 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":90,"threads_started":1}
I20260812 07:07:59.975734 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.967402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52440 (local address 127.31.156.67:35485)
0812 07:07:59.968664 (+ 1262us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.968678 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.968705 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:07:59.968965 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.968977 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.969222 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.969715 (+ 493us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:59.969905 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.971340 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.973388 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.974066 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.974232 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.974470 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.975431 (+ 961us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.975561 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1140,"thread_start_us":141,"threads_started":1}
I20260812 07:07:59.975786 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.967208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34199 (local address 127.0.0.1:36298)
0812 07:07:59.968176 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:59.968217 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:59.968894 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:59.969707 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:59.969710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:59.969730 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:59.969860 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.969869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.971209 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.971213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.973104 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.973119 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.974615 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.974621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.974825 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.974852 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 07:07:59.975534 (+ 682us) client_negotiation.cc:240] Negotiation successful
0812 07:07:59.975619 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":25,"spinlock_wait_cycles":5376,"thread_start_us":110,"threads_started":1}
I20260812 07:07:59.976603 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.968083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:43355 (local address 127.0.0.1:60480)
0812 07:07:59.968800 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:59.968817 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:59.968929 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:59.969453 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:59.969461 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:59.969510 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:59.969734 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.969744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.971508 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.971511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.974160 (+ 2649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:59.974186 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.976187 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.976191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:59.976294 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.976319 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:07:59.976402 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 07:07:59.976473 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":1298,"spinlock_wait_cycles":3072,"thread_start_us":91,"threads_started":1}
I20260812 07:07:59.977000 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.968541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.31.156.66:43355)
0812 07:07:59.968975 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.968991 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.969015 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:07:59.969088 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.969093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.969300 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.969457 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:59.969911 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.971373 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.974748 (+ 3375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.975762 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.975902 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.976116 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.976469 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.976588 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":132,"threads_started":1}
I20260812 07:07:59.979012 2357 tablet_service.cc:1467] Processing CreateTablet for tablet f2a6083de4264fdb9f959b30322102b3 (DEFAULT_TABLE table=test-table [id=307643ef446e43b28c61291e4ff05600]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:59.979012 2023 tablet_service.cc:1467] Processing CreateTablet for tablet f2a6083de4264fdb9f959b30322102b3 (DEFAULT_TABLE table=test-table [id=307643ef446e43b28c61291e4ff05600]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:59.980060 2357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2a6083de4264fdb9f959b30322102b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:59.980093 2023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2a6083de4264fdb9f959b30322102b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:59.979737 2196 tablet_service.cc:1467] Processing CreateTablet for tablet f2a6083de4264fdb9f959b30322102b3 (DEFAULT_TABLE table=test-table [id=307643ef446e43b28c61291e4ff05600]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:59.980660 2196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2a6083de4264fdb9f959b30322102b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:59.987824 2459 tablet_bootstrap.cc:492] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: Bootstrap starting.
I20260812 07:07:59.988970 2461 tablet_bootstrap.cc:492] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: Bootstrap starting.
I20260812 07:07:59.991061 2459 tablet_bootstrap.cc:654] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:59.991232 2460 tablet_bootstrap.cc:492] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: Bootstrap starting.
I20260812 07:07:59.991284 2461 tablet_bootstrap.cc:654] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:59.991992 2459 log.cc:824] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:59.992108 2461 log.cc:824] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:59.993759 2460 tablet_bootstrap.cc:654] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:59.994540 2459 tablet_bootstrap.cc:492] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: No bootstrap required, opened a new log
I20260812 07:07:59.994621 2460 log.cc:824] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:59.994787 2459 ts_tablet_manager.cc:1397] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:07:59.995283 2461 tablet_bootstrap.cc:492] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: No bootstrap required, opened a new log
I20260812 07:07:59.995503 2461 ts_tablet_manager.cc:1397] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 07:07:59.998754 2459 raft_consensus.cc:348] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } }
I20260812 07:07:59.999121 2459 raft_consensus.cc:374] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:59.999255 2459 raft_consensus.cc:729] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2302f3668514346910c50cadda3e7f5, State: Initialized, Role: FOLLOWER
I20260812 07:07:59.999262 2461 raft_consensus.cc:348] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } }
I20260812 07:07:59.999598 2461 raft_consensus.cc:374] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:59.999696 2461 raft_consensus.cc:729] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96eb3581fba04a3da45b174be1fa4940, State: Initialized, Role: FOLLOWER
I20260812 07:07:59.999778 2459 consensus_queue.cc:260] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } }
I20260812 07:08:00.000625 2461 consensus_queue.cc:260] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } }
I20260812 07:08:00.002135 2460 tablet_bootstrap.cc:492] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: No bootstrap required, opened a new log
I20260812 07:08:00.002840 2460 ts_tablet_manager.cc:1397] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260812 07:08:00.002768 2275 heartbeater.cc:503] Master 127.31.156.126:41753 was elected leader, sending a full tablet report...
I20260812 07:08:00.002768 2461 ts_tablet_manager.cc:1428] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260812 07:08:00.003614 2459 ts_tablet_manager.cc:1428] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260812 07:08:00.004101 2110 heartbeater.cc:503] Master 127.31.156.126:41753 was elected leader, sending a full tablet report...
I20260812 07:08:00.008639 2460 raft_consensus.cc:348] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } }
I20260812 07:08:00.009032 2460 raft_consensus.cc:374] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:00.009078 2460 raft_consensus.cc:729] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9f5e32109f478f8493792225db338b, State: Initialized, Role: FOLLOWER
I20260812 07:08:00.009588 2460 consensus_queue.cc:260] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } }
I20260812 07:08:00.010416 2432 heartbeater.cc:503] Master 127.31.156.126:41753 was elected leader, sending a full tablet report...
I20260812 07:08:00.010792 2460 ts_tablet_manager.cc:1428] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260812 07:08:00.116278 2111 tablet.cc:2369] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:08:00.146857 2276 tablet.cc:2369] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:08:00.154037 2433 tablet.cc:2369] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:08:00.221877 2465 raft_consensus.cc:482] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:00.222177 2465 raft_consensus.cc:504] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } }
I20260812 07:08:00.223688 2465 leader_election.cc:288] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485), 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355)
I20260812 07:08:00.228444 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.224225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:46579 (local address 127.31.156.67:35485)
0812 07:08:00.224513 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:00.224519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:00.224653 (+ 134us) server_negotiation.cc:407] Connection header received
0812 07:08:00.224856 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:00.224860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:00.224924 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:00.225043 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:00.225384 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.225981 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.227279 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.227791 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.227943 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.228006 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:00.228206 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 07:08:00.228281 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:08:00.228458 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.224749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:38943 (local address 127.31.156.66:43355)
0812 07:08:00.224996 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:00.225002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:00.225177 (+ 175us) server_negotiation.cc:407] Connection header received
0812 07:08:00.225345 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:00.225353 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:00.225395 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:00.225482 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:00.225762 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.226265 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.227401 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.227842 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.227986 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.228044 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:00.228240 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 07:08:00.228328 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260812 07:08:00.228462 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.224029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35485 (local address 127.31.156.65:46579)
0812 07:08:00.224518 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:00.224536 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:00.224720 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:00.225062 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:00.225065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:00.225087 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:00.225245 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.225251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.226110 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.226117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.227131 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.227142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.227944 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.227948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.228062 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.228092 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:08:00.228187 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 07:08:00.228274 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":157,"threads_started":1}
I20260812 07:08:00.228470 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.224635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:43355 (local address 127.31.156.65:38943)
0812 07:08:00.225055 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:00.225067 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:00.225219 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:00.225495 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:00.225497 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:00.225515 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:00.225642 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.225648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.226419 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.226424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.227261 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.227271 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.227986 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.227990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.228115 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.228134 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 07:08:00.228223 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 07:08:00.228297 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":107,"threads_started":1}
I20260812 07:08:00.229277 2219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a6083de4264fdb9f959b30322102b3" candidate_uuid: "c2302f3668514346910c50cadda3e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96eb3581fba04a3da45b174be1fa4940" is_pre_election: true
I20260812 07:08:00.229293 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a6083de4264fdb9f959b30322102b3" candidate_uuid: "c2302f3668514346910c50cadda3e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9f5e32109f478f8493792225db338b" is_pre_election: true
I20260812 07:08:00.229642 2380 raft_consensus.cc:2454] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2302f3668514346910c50cadda3e7f5 in term 0.
I20260812 07:08:00.229641 2219 raft_consensus.cc:2454] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2302f3668514346910c50cadda3e7f5 in term 0.
I20260812 07:08:00.230297 1981 leader_election.cc:302] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96eb3581fba04a3da45b174be1fa4940, c2302f3668514346910c50cadda3e7f5; no voters:
I20260812 07:08:00.230691 2465 raft_consensus.cc:2781] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:00.230783 2465 raft_consensus.cc:482] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:00.230819 2465 raft_consensus.cc:3037] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:00.232283 2465 raft_consensus.cc:504] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } }
I20260812 07:08:00.232720 2465 leader_election.cc:288] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [CANDIDATE]: Term 1 election: Requested vote from peers 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485), 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355)
I20260812 07:08:00.233299 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a6083de4264fdb9f959b30322102b3" candidate_uuid: "c2302f3668514346910c50cadda3e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9f5e32109f478f8493792225db338b"
I20260812 07:08:00.233453 2219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a6083de4264fdb9f959b30322102b3" candidate_uuid: "c2302f3668514346910c50cadda3e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96eb3581fba04a3da45b174be1fa4940"
I20260812 07:08:00.233513 2380 raft_consensus.cc:3037] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:00.233640 2219 raft_consensus.cc:3037] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:00.234925 2219 raft_consensus.cc:2454] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2302f3668514346910c50cadda3e7f5 in term 1.
I20260812 07:08:00.235239 2380 raft_consensus.cc:2454] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2302f3668514346910c50cadda3e7f5 in term 1.
I20260812 07:08:00.235456 1981 leader_election.cc:302] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96eb3581fba04a3da45b174be1fa4940, c2302f3668514346910c50cadda3e7f5; no voters:
I20260812 07:08:00.235690 2465 raft_consensus.cc:2781] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:00.236016 2465 raft_consensus.cc:686] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 1 LEADER]: Becoming Leader. State: Replica: c2302f3668514346910c50cadda3e7f5, State: Running, Role: LEADER
I20260812 07:08:00.236428 2465 consensus_queue.cc:237] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } }
I20260812 07:08:00.240466 1877 catalog_manager.cc:5495] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2302f3668514346910c50cadda3e7f5 (127.31.156.65). New cstate: current_term: 1 leader_uuid: "c2302f3668514346910c50cadda3e7f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } health_report { overall_health: HEALTHY } } }
Aug 12 07:08:00 dist-test-slave-csg5 krb5kdc[1829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518478, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:00.300196 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.288963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41753 (local address 127.0.0.1:43290)
0812 07:08:00.289462 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:00.289476 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:00.289642 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:00.289942 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:00.289945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:00.290499 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:00.290613 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.290621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.291581 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.291585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.292309 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.292327 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.293133 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.293137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.293220 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.293962 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:00.293977 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:00.295959 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:00.298173 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:00.298190 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:00.298198 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:00.298498 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:00.298889 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:00.298892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:00.298893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:00.298983 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:00.299504 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:00.299515 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:00.299645 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 07:08:00.299772 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 07:08:00.299980 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":170,"threads_started":1}
I20260812 07:08:00.300196 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.289089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.31.156.126:41753)
0812 07:08:00.289288 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:00.289295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:00.289569 (+ 274us) server_negotiation.cc:407] Connection header received
0812 07:08:00.289762 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:00.289765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:00.289822 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:00.289917 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:00.290797 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.291437 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.292457 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.292987 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.293127 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.296144 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:00.296147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:00.296149 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:00.296174 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:00.297987 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:00.298609 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:00.298614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:00.298616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:00.298765 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:00.299075 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:00.299079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:00.299081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:00.299382 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:00.299504 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:00.299836 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 07:08:00.300050 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":67}
I20260812 07:08:00.305735 1877 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:43290:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 07:08:00.306581 1877 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:00.312520 2023 tablet_service.cc:1467] Processing CreateTablet for tablet 2ede0378d6494b9492a2c56cb9fe6854 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5bcf39ad73fe4fe8b9e4cf66eb1fb3fa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:08:00.312510 2196 tablet_service.cc:1467] Processing CreateTablet for tablet 2ede0378d6494b9492a2c56cb9fe6854 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5bcf39ad73fe4fe8b9e4cf66eb1fb3fa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:08:00.313390 2023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ede0378d6494b9492a2c56cb9fe6854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:00.313390 2196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ede0378d6494b9492a2c56cb9fe6854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:00.315531 2357 tablet_service.cc:1467] Processing CreateTablet for tablet 2ede0378d6494b9492a2c56cb9fe6854 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5bcf39ad73fe4fe8b9e4cf66eb1fb3fa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:08:00.316531 2357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ede0378d6494b9492a2c56cb9fe6854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:00.316711 2461 tablet_bootstrap.cc:492] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940: Bootstrap starting.
I20260812 07:08:00.316850 2459 tablet_bootstrap.cc:492] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5: Bootstrap starting.
I20260812 07:08:00.317873 2459 tablet_bootstrap.cc:654] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:00.317873 2461 tablet_bootstrap.cc:654] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:00.320184 2460 tablet_bootstrap.cc:492] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: Bootstrap starting.
I20260812 07:08:00.320190 2461 tablet_bootstrap.cc:492] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940: No bootstrap required, opened a new log
I20260812 07:08:00.320294 2459 tablet_bootstrap.cc:492] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5: No bootstrap required, opened a new log
I20260812 07:08:00.320310 2461 ts_tablet_manager.cc:1397] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 07:08:00.320403 2459 ts_tablet_manager.cc:1397] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 07:08:00.320881 2461 raft_consensus.cc:348] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.320911 2459 raft_consensus.cc:348] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.321015 2459 raft_consensus.cc:374] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:00.321015 2461 raft_consensus.cc:374] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:00.321040 2459 raft_consensus.cc:729] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2302f3668514346910c50cadda3e7f5, State: Initialized, Role: FOLLOWER
I20260812 07:08:00.321048 2461 raft_consensus.cc:729] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96eb3581fba04a3da45b174be1fa4940, State: Initialized, Role: FOLLOWER
I20260812 07:08:00.321204 2459 consensus_queue.cc:260] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.321214 2461 consensus_queue.cc:260] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.321404 2460 tablet_bootstrap.cc:654] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:00.321501 2465 tablet_replica.cc:439] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } }
I20260812 07:08:00.321646 2465 tablet_replica.cc:442] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:00.321566 2466 tablet_replica.cc:439] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } }
I20260812 07:08:00.321707 2466 tablet_replica.cc:442] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:00.321889 2459 ts_tablet_manager.cc:1428] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 07:08:00.322758 2461 ts_tablet_manager.cc:1428] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:08:00.323026 2460 tablet_bootstrap.cc:492] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: No bootstrap required, opened a new log
I20260812 07:08:00.323144 2460 ts_tablet_manager.cc:1397] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:00.323649 2460 raft_consensus.cc:348] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.323820 2460 raft_consensus.cc:374] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:00.323864 2460 raft_consensus.cc:729] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9f5e32109f478f8493792225db338b, State: Initialized, Role: FOLLOWER
I20260812 07:08:00.324077 2460 consensus_queue.cc:260] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.324450 2460 ts_tablet_manager.cc:1428] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:08:00.324563 2468 tablet_replica.cc:439] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } }
I20260812 07:08:00.324741 2468 tablet_replica.cc:442] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:00.345572 2468 raft_consensus.cc:482] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:00.345789 2468 raft_consensus.cc:504] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.346589 2466 raft_consensus.cc:482] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:00.346877 2466 raft_consensus.cc:504] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.347424 2468 leader_election.cc:288] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199), 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355)
I20260812 07:08:00.348251 2466 leader_election.cc:288] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199), 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485)
I20260812 07:08:00.354847 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.349315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:53011 (local address 127.31.156.67:35485)
0812 07:08:00.349639 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:00.349646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:00.349664 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:08:00.349715 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:00.349719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:00.349773 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:00.349858 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:00.350303 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.351377 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.353259 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.353949 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.354054 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.354113 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:00.354555 (+ 442us) server_negotiation.cc:299] Negotiation successful
0812 07:08:00.354677 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235}
I20260812 07:08:00.357251 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.348635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35485 (local address 127.31.156.66:53011)
0812 07:08:00.348992 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:00.349009 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:00.349125 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:00.349866 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:00.349870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:00.349885 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:00.350025 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.350033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.351571 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.351574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.353084 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.353097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.354256 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.354260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.354373 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.354398 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:08:00.356940 (+ 2542us) client_negotiation.cc:240] Negotiation successful
0812 07:08:00.357042 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260812 07:08:00.358150 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ede0378d6494b9492a2c56cb9fe6854" candidate_uuid: "96eb3581fba04a3da45b174be1fa4940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9f5e32109f478f8493792225db338b" is_pre_election: true
I20260812 07:08:00.358336 2380 raft_consensus.cc:2454] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96eb3581fba04a3da45b174be1fa4940 in term 0.
I20260812 07:08:00.358572 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.347682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34199 (local address 127.31.156.67:33051)
0812 07:08:00.347824 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:00.347844 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:00.347951 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:00.353356 (+ 5405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:00.353359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:00.353385 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:00.353510 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.353517 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.354626 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.354630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.356617 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.356636 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.358037 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.358044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.358182 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.358211 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:08:00.358297 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 07:08:00.358392 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:08:00.359030 2145 leader_election.cc:302] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9f5e32109f478f8493792225db338b, 96eb3581fba04a3da45b174be1fa4940; no voters:
I20260812 07:08:00.359458 2466 raft_consensus.cc:2781] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:00.359532 2466 raft_consensus.cc:482] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:00.359581 2466 raft_consensus.cc:3037] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:00.359822 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.348032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:33051 (local address 127.31.156.65:34199)
0812 07:08:00.348250 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:00.348256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:00.348276 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:08:00.348369 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:00.348375 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:00.348443 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:00.348548 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:00.353658 (+ 5110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.354463 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.356830 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.357624 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.357780 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.357846 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:00.359526 (+ 1680us) server_negotiation.cc:299] Negotiation successful
0812 07:08:00.359650 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":119}
I20260812 07:08:00.360567 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.349349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:37343 (local address 127.31.156.65:34199)
0812 07:08:00.350345 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:00.350361 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:00.350378 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:08:00.350432 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:00.350435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:00.350485 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:00.350598 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:00.351665 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.352595 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.356871 (+ 4276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.358805 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.358924 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.358990 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:00.359676 (+ 686us) server_negotiation.cc:299] Negotiation successful
0812 07:08:00.359785 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":136,"threads_started":1}
I20260812 07:08:00.360877 2053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ede0378d6494b9492a2c56cb9fe6854" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2302f3668514346910c50cadda3e7f5" is_pre_election: true
I20260812 07:08:00.361073 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.348945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34199 (local address 127.31.156.66:37343)
0812 07:08:00.349586 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:00.349615 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:00.349715 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:00.350755 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:00.350758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:00.350780 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:00.351425 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.351440 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.352774 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.352778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.356648 (+ 3870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.356662 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.359186 (+ 2524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.359219 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.359415 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.359446 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:08:00.359842 (+ 396us) client_negotiation.cc:240] Negotiation successful
0812 07:08:00.360907 (+ 1065us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":1970,"thread_start_us":109,"threads_started":1}
I20260812 07:08:00.361168 2053 raft_consensus.cc:2454] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d9f5e32109f478f8493792225db338b in term 0.
I20260812 07:08:00.361238 2466 raft_consensus.cc:504] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.361652 2466 leader_election.cc:288] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [CANDIDATE]: Term 1 election: Requested vote from peers c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199), 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485)
I20260812 07:08:00.361756 2305 leader_election.cc:302] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9f5e32109f478f8493792225db338b, c2302f3668514346910c50cadda3e7f5; no voters:
I20260812 07:08:00.362138 2468 raft_consensus.cc:2781] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:00.362221 2468 raft_consensus.cc:482] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:00.362293 2468 raft_consensus.cc:3037] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:00.362375 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ede0378d6494b9492a2c56cb9fe6854" candidate_uuid: "96eb3581fba04a3da45b174be1fa4940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9f5e32109f478f8493792225db338b"
I20260812 07:08:00.362799 2053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ede0378d6494b9492a2c56cb9fe6854" candidate_uuid: "96eb3581fba04a3da45b174be1fa4940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2302f3668514346910c50cadda3e7f5" is_pre_election: true
I20260812 07:08:00.362938 2053 raft_consensus.cc:2454] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96eb3581fba04a3da45b174be1fa4940 in term 0.
I20260812 07:08:00.363837 2468 raft_consensus.cc:504] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.364187 2380 raft_consensus.cc:2379] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 96eb3581fba04a3da45b174be1fa4940 in current term 1: Already voted for candidate 3d9f5e32109f478f8493792225db338b in this term.
I20260812 07:08:00.364358 2468 leader_election.cc:288] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 election: Requested vote from peers c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199), 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355)
I20260812 07:08:00.364818 2053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ede0378d6494b9492a2c56cb9fe6854" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2302f3668514346910c50cadda3e7f5"
I20260812 07:08:00.364956 2053 raft_consensus.cc:3037] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:00.366257 2053 raft_consensus.cc:2454] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d9f5e32109f478f8493792225db338b in term 1.
I20260812 07:08:00.366731 2305 leader_election.cc:302] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9f5e32109f478f8493792225db338b, c2302f3668514346910c50cadda3e7f5; no voters:
I20260812 07:08:00.366914 2468 raft_consensus.cc:2781] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:00.367173 2468 raft_consensus.cc:686] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [term 1 LEADER]: Becoming Leader. State: Replica: 3d9f5e32109f478f8493792225db338b, State: Running, Role: LEADER
I20260812 07:08:00.367266 2052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ede0378d6494b9492a2c56cb9fe6854" candidate_uuid: "96eb3581fba04a3da45b174be1fa4940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2302f3668514346910c50cadda3e7f5"
I20260812 07:08:00.367463 2052 raft_consensus.cc:2379] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 96eb3581fba04a3da45b174be1fa4940 in current term 1: Already voted for candidate 3d9f5e32109f478f8493792225db338b in this term.
I20260812 07:08:00.367609 2468 consensus_queue.cc:237] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.369923 2490 tablet_replica.cc:439] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: TxnStatusTablet state changed. Reason: New leader 3d9f5e32109f478f8493792225db338b. Latest consensus state: current_term: 1 leader_uuid: "3d9f5e32109f478f8493792225db338b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } }
I20260812 07:08:00.370059 2490 tablet_replica.cc:442] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:08:00.376194 2144 leader_election.cc:302] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96eb3581fba04a3da45b174be1fa4940; no voters: 3d9f5e32109f478f8493792225db338b, c2302f3668514346910c50cadda3e7f5
I20260812 07:08:00.376286 2492 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 07:08:00.376525 2492 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 07:08:00.376767 1876 catalog_manager.cc:5495] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d9f5e32109f478f8493792225db338b (127.31.156.67). New cstate: current_term: 1 leader_uuid: "3d9f5e32109f478f8493792225db338b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } health_report { overall_health: HEALTHY } } }
I20260812 07:08:00.377552 2466 raft_consensus.cc:2726] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 07:08:00.388839 2484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.347684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:43355 (local address 127.31.156.67:32861)
0812 07:08:00.353027 (+ 5343us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:00.353047 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:00.353180 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:00.358735 (+ 5555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:00.358740 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:00.358770 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:00.358907 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.358915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.385658 (+ 26743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.385662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.387225 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.387244 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.388261 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.388266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.388404 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.388429 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:08:00.388536 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 07:08:00.388639 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5239,"mutex_wait_us":261,"thread_start_us":271,"threads_started":1}
I20260812 07:08:00.388805 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.347807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:32861 (local address 127.31.156.66:43355)
0812 07:08:00.347937 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:00.347943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:00.353387 (+ 5444us) server_negotiation.cc:407] Connection header received
0812 07:08:00.353460 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:00.353466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:00.353534 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:00.353633 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:00.362531 (+ 8898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.363385 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.387433 (+ 24048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.388079 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.388244 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.388310 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:00.388574 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 07:08:00.388662 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:08:00.389956 2219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ede0378d6494b9492a2c56cb9fe6854" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96eb3581fba04a3da45b174be1fa4940" is_pre_election: true
I20260812 07:08:00.390221 2219 raft_consensus.cc:2379] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d9f5e32109f478f8493792225db338b in current term 1: Already voted for candidate 96eb3581fba04a3da45b174be1fa4940 in this term.
I20260812 07:08:00.390242 2217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ede0378d6494b9492a2c56cb9fe6854" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96eb3581fba04a3da45b174be1fa4940"
I20260812 07:08:00.390472 2217 raft_consensus.cc:2379] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d9f5e32109f478f8493792225db338b in current term 1: Already voted for candidate 96eb3581fba04a3da45b174be1fa4940 in this term.
I20260812 07:08:00.401568 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.395429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35485 (local address 127.0.0.1:52450)
0812 07:08:00.395781 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:00.395797 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:00.395922 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:00.396367 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:00.396370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:00.396395 (+ 25us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:08:00.396510 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.396516 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.397864 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.397870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.399072 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.399091 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.399870 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.399874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.400117 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.400193 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:08:00.401227 (+ 1034us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:08:00.401247 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 07:08:00.401323 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 07:08:00.401431 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260812 07:08:00.401600 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.395571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.31.156.67:35485)
0812 07:08:00.395756 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:00.395762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:00.395853 (+ 91us) server_negotiation.cc:407] Connection header received
0812 07:08:00.396090 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:00.396095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:00.396180 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:00.396317 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:00.396647 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.397552 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.399236 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.399653 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.399787 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.400446 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:00.401078 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:00.401212 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:00.401365 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 07:08:00.401441 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 07:08:00.409025 2052 raft_consensus.cc:1261] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Refusing update from remote peer 3d9f5e32109f478f8493792225db338b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:00.409149 2217 raft_consensus.cc:1261] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Refusing update from remote peer 3d9f5e32109f478f8493792225db338b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:00.409824 2490 consensus_queue.cc:1035] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:00.410104 2468 consensus_queue.cc:1035] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b [LEADER]: Connected to new peer: Peer: permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:00.414149 2465 tablet_replica.cc:439] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5: TxnStatusTablet state changed. Reason: New leader 3d9f5e32109f478f8493792225db338b. Latest consensus state: current_term: 1 leader_uuid: "3d9f5e32109f478f8493792225db338b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } }
I20260812 07:08:00.414310 2465 tablet_replica.cc:442] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:00.415095 2466 tablet_replica.cc:439] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940: TxnStatusTablet state changed. Reason: New leader 3d9f5e32109f478f8493792225db338b. Latest consensus state: current_term: 1 leader_uuid: "3d9f5e32109f478f8493792225db338b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } }
I20260812 07:08:00.415235 2466 tablet_replica.cc:442] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:00.419600 2490 tablet_replica.cc:439] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d9f5e32109f478f8493792225db338b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } }
I20260812 07:08:00.419775 2490 tablet_replica.cc:442] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:08:00.419828 2466 tablet_replica.cc:439] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d9f5e32109f478f8493792225db338b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } }
I20260812 07:08:00.419973 2466 tablet_replica.cc:442] T 2ede0378d6494b9492a2c56cb9fe6854 P 96eb3581fba04a3da45b174be1fa4940: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:00.421089 2497 mvcc.cc:205] Tried to move back new op lower bound from 7317579695747158016 to 7317579695589175296. Current Snapshot: MvccSnapshot[applied={T|T < 7317579695747158016}]
I20260812 07:08:00.421135 2468 tablet_replica.cc:439] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d9f5e32109f478f8493792225db338b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } }
I20260812 07:08:00.421290 2468 tablet_replica.cc:442] T 2ede0378d6494b9492a2c56cb9fe6854 P 3d9f5e32109f478f8493792225db338b: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:08:00.423841 2465 tablet_replica.cc:439] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d9f5e32109f478f8493792225db338b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } } }
I20260812 07:08:00.424037 2465 tablet_replica.cc:442] T 2ede0378d6494b9492a2c56cb9fe6854 P c2302f3668514346910c50cadda3e7f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:00.425845 2495 mvcc.cc:205] Tried to move back new op lower bound from 7317579695747158016 to 7317579695589175296. Current Snapshot: MvccSnapshot[applied={T|T < 7317579695747158016 or (T in {7317579695747158016})}]
I20260812 07:08:00.433497 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.428471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36306 (local address 127.31.156.65:34199)
0812 07:08:00.428629 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:00.428634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:00.428652 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:08:00.428813 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:00.428815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:00.428873 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:00.428973 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:00.429323 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.430056 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.431461 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.431739 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.431843 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.432299 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:00.432830 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:00.432943 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:00.433082 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 07:08:00.433290 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:08:00.448302 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.443483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:47731 (local address 127.31.156.67:35485)
0812 07:08:00.443640 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:00.443645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:00.443932 (+ 287us) server_negotiation.cc:407] Connection header received
0812 07:08:00.444077 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:00.444080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:00.444140 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:00.444239 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:00.444655 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.445611 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.446838 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.447191 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.447313 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.447708 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:00.447827 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:00.447916 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:00.448071 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 07:08:00.448136 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:08:00.448367 2507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.443405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35485 (local address 127.31.156.65:47731)
0812 07:08:00.443823 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:00.443846 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:00.443967 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:00.444386 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:00.444389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:00.444397 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:08:00.444524 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.444529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.445785 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.445788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.446622 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.446633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.447353 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.447357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.447460 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.447520 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:08:00.447952 (+ 432us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:08:00.447959 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:08:00.448029 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 07:08:00.448125 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":185,"threads_started":1}
I20260812 07:08:00.458076 2217 raft_consensus.cc:1261] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Refusing update from remote peer c2302f3668514346910c50cadda3e7f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:00.458094 2380 raft_consensus.cc:1261] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Refusing update from remote peer c2302f3668514346910c50cadda3e7f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:00.458743 2471 consensus_queue.cc:1035] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:00.458967 2465 consensus_queue.cc:1035] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:00.463650 2496 mvcc.cc:205] Tried to move back new op lower bound from 7317579695950135296 to 7317579695051661312. Current Snapshot: MvccSnapshot[applied={T|T < 7317579695950135296}]
I20260812 07:08:00.489158 2513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.484304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34199 (local address 127.31.156.67:48249)
0812 07:08:00.484733 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:00.484751 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:00.484862 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:00.485251 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:00.485255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:00.485264 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:08:00.485396 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.485404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.486554 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.486558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.487500 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:00.487511 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.488297 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.488301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:00.488376 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.488441 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:08:00.488835 (+ 394us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:08:00.488843 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:08:00.488918 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 07:08:00.489000 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":125,"threads_started":1}
I20260812 07:08:00.489482 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:00.484721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:48249 (local address 127.31.156.65:34199)
0812 07:08:00.484992 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:00.484997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:00.485011 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:08:00.485065 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:00.485068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:00.485129 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:00.485237 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:00.485536 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.486436 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.487632 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:00.488033 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:00.488152 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:00.488593 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:00.488690 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:00.488793 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:00.489251 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 07:08:00.489336 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260812 07:08:00.528578 1874 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:00.528823 1874 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:00.532490 1874 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76: Sending DeleteTablet for 3 replicas of tablet f2a6083de4264fdb9f959b30322102b3
I20260812 07:08:00.533581 2023 tablet_service.cc:1514] Processing DeleteTablet for tablet f2a6083de4264fdb9f959b30322102b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:00 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:36298
I20260812 07:08:00.533581 2196 tablet_service.cc:1514] Processing DeleteTablet for tablet f2a6083de4264fdb9f959b30322102b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:00 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:60480
I20260812 07:08:00.534715 2357 tablet_service.cc:1514] Processing DeleteTablet for tablet f2a6083de4264fdb9f959b30322102b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:00 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:52440
I20260812 07:08:00.535392 2519 tablet_replica.cc:330] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: stopping tablet replica
I20260812 07:08:00.535445 1876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:08:00.535992 1876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:00.536055 2521 tablet_replica.cc:330] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: stopping tablet replica
I20260812 07:08:00.536337 2519 raft_consensus.cc:2229] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:00.536381 2521 raft_consensus.cc:2229] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:00.536599 2521 raft_consensus.cc:2258] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:00.536871 2519 raft_consensus.cc:2258] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:00.539197 2519 ts_tablet_manager.cc:1905] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:00.543615 2023 tablet_service.cc:1467] Processing CreateTablet for tablet 99957fb0c9db47de82ec31a121d7fc7b (DEFAULT_TABLE table=test-table [id=643b9e8d11a042219fc26dd9af62f18b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:00.543761 2196 tablet_service.cc:1467] Processing CreateTablet for tablet 99957fb0c9db47de82ec31a121d7fc7b (DEFAULT_TABLE table=test-table [id=643b9e8d11a042219fc26dd9af62f18b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:00.543982 2023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99957fb0c9db47de82ec31a121d7fc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:00.544085 2196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99957fb0c9db47de82ec31a121d7fc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:00.547186 2357 tablet_service.cc:1467] Processing CreateTablet for tablet 99957fb0c9db47de82ec31a121d7fc7b (DEFAULT_TABLE table=test-table [id=643b9e8d11a042219fc26dd9af62f18b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:00.547565 2357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99957fb0c9db47de82ec31a121d7fc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:00.548899 2459 tablet_bootstrap.cc:492] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5: Bootstrap starting.
I20260812 07:08:00.550225 2459 tablet_bootstrap.cc:654] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:00.551081 2461 tablet_bootstrap.cc:492] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940: Bootstrap starting.
I20260812 07:08:00.551744 2521 ts_tablet_manager.cc:1905] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:00.552445 2461 tablet_bootstrap.cc:654] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:00.552943 2520 tablet_replica.cc:330] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: stopping tablet replica
I20260812 07:08:00.553347 2520 raft_consensus.cc:2229] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:00.554093 2460 tablet_bootstrap.cc:492] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b: Bootstrap starting.
I20260812 07:08:00.553829 2520 raft_consensus.cc:2258] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:00.555912 2521 ts_tablet_manager.cc:1918] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:08:00.555959 2460 tablet_bootstrap.cc:654] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:00.556727 2521 log.cc:1196] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/f2a6083de4264fdb9f959b30322102b3
I20260812 07:08:00.556718 2520 ts_tablet_manager.cc:1905] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:00.557164 2521 ts_tablet_manager.cc:1939] T f2a6083de4264fdb9f959b30322102b3 P 3d9f5e32109f478f8493792225db338b: Deleting consensus metadata
I20260812 07:08:00.557720 2519 ts_tablet_manager.cc:1918] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:08:00.557945 2519 log.cc:1196] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/f2a6083de4264fdb9f959b30322102b3
I20260812 07:08:00.558455 2519 ts_tablet_manager.cc:1939] T f2a6083de4264fdb9f959b30322102b3 P 96eb3581fba04a3da45b174be1fa4940: Deleting consensus metadata
I20260812 07:08:00.558738 1864 catalog_manager.cc:4841] TS 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485): tablet f2a6083de4264fdb9f959b30322102b3 (table test-table [id=307643ef446e43b28c61291e4ff05600]) successfully deleted
I20260812 07:08:00.560545 1864 catalog_manager.cc:4841] TS 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355): tablet f2a6083de4264fdb9f959b30322102b3 (table test-table [id=307643ef446e43b28c61291e4ff05600]) successfully deleted
I20260812 07:08:00.561973 2461 tablet_bootstrap.cc:492] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940: No bootstrap required, opened a new log
I20260812 07:08:00.562095 2461 ts_tablet_manager.cc:1397] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260812 07:08:00.563105 2520 ts_tablet_manager.cc:1918] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:08:00.563059 2461 raft_consensus.cc:348] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.563232 2520 log.cc:1196] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/wal/wals/f2a6083de4264fdb9f959b30322102b3
I20260812 07:08:00.563246 2461 raft_consensus.cc:374] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:00.563289 2461 raft_consensus.cc:729] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96eb3581fba04a3da45b174be1fa4940, State: Initialized, Role: FOLLOWER
I20260812 07:08:00.563444 2461 consensus_queue.cc:260] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.563512 2459 tablet_bootstrap.cc:492] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5: No bootstrap required, opened a new log
I20260812 07:08:00.563591 2520 ts_tablet_manager.cc:1939] T f2a6083de4264fdb9f959b30322102b3 P c2302f3668514346910c50cadda3e7f5: Deleting consensus metadata
I20260812 07:08:00.563619 2459 ts_tablet_manager.cc:1397] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.001s
I20260812 07:08:00.563786 2461 ts_tablet_manager.cc:1428] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 07:08:00.564147 2459 raft_consensus.cc:348] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.564282 2459 raft_consensus.cc:374] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:00.564318 2459 raft_consensus.cc:729] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2302f3668514346910c50cadda3e7f5, State: Initialized, Role: FOLLOWER
I20260812 07:08:00.564455 2459 consensus_queue.cc:260] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.564767 2459 ts_tablet_manager.cc:1428] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 07:08:00.565590 2460 tablet_bootstrap.cc:492] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b: No bootstrap required, opened a new log
I20260812 07:08:00.565709 2460 ts_tablet_manager.cc:1397] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260812 07:08:00.566270 2460 raft_consensus.cc:348] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.566427 2460 raft_consensus.cc:374] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:00.566483 2460 raft_consensus.cc:729] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9f5e32109f478f8493792225db338b, State: Initialized, Role: FOLLOWER
I20260812 07:08:00.566627 2460 consensus_queue.cc:260] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.567004 2460 ts_tablet_manager.cc:1428] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 07:08:00.572175 1863 catalog_manager.cc:4841] TS c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199): tablet f2a6083de4264fdb9f959b30322102b3 (table test-table [id=307643ef446e43b28c61291e4ff05600]) successfully deleted
I20260812 07:08:00.986052 2508 raft_consensus.cc:482] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:00.986363 2508 raft_consensus.cc:504] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.986971 2508 leader_election.cc:288] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199), 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355)
I20260812 07:08:00.987516 2219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99957fb0c9db47de82ec31a121d7fc7b" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96eb3581fba04a3da45b174be1fa4940" is_pre_election: true
I20260812 07:08:00.987519 2052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99957fb0c9db47de82ec31a121d7fc7b" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2302f3668514346910c50cadda3e7f5" is_pre_election: true
I20260812 07:08:00.987707 2052 raft_consensus.cc:2454] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d9f5e32109f478f8493792225db338b in term 0.
I20260812 07:08:00.987748 2219 raft_consensus.cc:2454] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d9f5e32109f478f8493792225db338b in term 0.
I20260812 07:08:00.987875 2466 raft_consensus.cc:482] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:00.987983 2466 raft_consensus.cc:504] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.988142 2306 leader_election.cc:302] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9f5e32109f478f8493792225db338b, 96eb3581fba04a3da45b174be1fa4940; no voters:
I20260812 07:08:00.988379 2466 leader_election.cc:288] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199), 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485)
I20260812 07:08:00.988557 2508 raft_consensus.cc:2781] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:00.988641 2508 raft_consensus.cc:482] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:00.988705 2508 raft_consensus.cc:3037] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:00.988745 2052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99957fb0c9db47de82ec31a121d7fc7b" candidate_uuid: "96eb3581fba04a3da45b174be1fa4940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2302f3668514346910c50cadda3e7f5" is_pre_election: true
I20260812 07:08:00.988745 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99957fb0c9db47de82ec31a121d7fc7b" candidate_uuid: "96eb3581fba04a3da45b174be1fa4940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9f5e32109f478f8493792225db338b" is_pre_election: true
I20260812 07:08:00.988881 2052 raft_consensus.cc:2454] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96eb3581fba04a3da45b174be1fa4940 in term 0.
I20260812 07:08:00.989183 2144 leader_election.cc:302] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96eb3581fba04a3da45b174be1fa4940, c2302f3668514346910c50cadda3e7f5; no voters:
I20260812 07:08:00.989372 2466 raft_consensus.cc:2781] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:00.989445 2466 raft_consensus.cc:482] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:00.989494 2466 raft_consensus.cc:3037] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:00.990283 2508 raft_consensus.cc:504] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.990547 2380 raft_consensus.cc:2379] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96eb3581fba04a3da45b174be1fa4940 in current term 1: Already voted for candidate 3d9f5e32109f478f8493792225db338b in this term.
I20260812 07:08:00.990744 2508 leader_election.cc:288] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 election: Requested vote from peers c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199), 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355)
I20260812 07:08:00.990936 2466 raft_consensus.cc:504] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.991214 2219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99957fb0c9db47de82ec31a121d7fc7b" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96eb3581fba04a3da45b174be1fa4940"
I20260812 07:08:00.991281 2052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99957fb0c9db47de82ec31a121d7fc7b" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2302f3668514346910c50cadda3e7f5"
I20260812 07:08:00.991344 2466 leader_election.cc:288] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [CANDIDATE]: Term 1 election: Requested vote from peers c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199), 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485)
I20260812 07:08:00.991371 2219 raft_consensus.cc:2379] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d9f5e32109f478f8493792225db338b in current term 1: Already voted for candidate 96eb3581fba04a3da45b174be1fa4940 in this term.
I20260812 07:08:00.991402 2052 raft_consensus.cc:3037] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:00.991744 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99957fb0c9db47de82ec31a121d7fc7b" candidate_uuid: "96eb3581fba04a3da45b174be1fa4940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9f5e32109f478f8493792225db338b"
I20260812 07:08:00.991890 2380 raft_consensus.cc:2379] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 96eb3581fba04a3da45b174be1fa4940 in current term 1: Already voted for candidate 3d9f5e32109f478f8493792225db338b in this term.
I20260812 07:08:00.992069 2053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99957fb0c9db47de82ec31a121d7fc7b" candidate_uuid: "96eb3581fba04a3da45b174be1fa4940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2302f3668514346910c50cadda3e7f5"
I20260812 07:08:00.992475 2144 leader_election.cc:302] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96eb3581fba04a3da45b174be1fa4940; no voters: 3d9f5e32109f478f8493792225db338b, c2302f3668514346910c50cadda3e7f5
I20260812 07:08:00.992648 2466 raft_consensus.cc:2726] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 07:08:00.992868 2052 raft_consensus.cc:2454] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d9f5e32109f478f8493792225db338b in term 1.
I20260812 07:08:00.993175 2305 leader_election.cc:302] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d9f5e32109f478f8493792225db338b, c2302f3668514346910c50cadda3e7f5; no voters: 96eb3581fba04a3da45b174be1fa4940
I20260812 07:08:00.993362 2508 raft_consensus.cc:2781] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:00.993465 2508 raft_consensus.cc:686] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 1 LEADER]: Becoming Leader. State: Replica: 3d9f5e32109f478f8493792225db338b, State: Running, Role: LEADER
I20260812 07:08:00.993623 2508 consensus_queue.cc:237] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:00.995518 1877 catalog_manager.cc:5495] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d9f5e32109f478f8493792225db338b (127.31.156.67). New cstate: current_term: 1 leader_uuid: "3d9f5e32109f478f8493792225db338b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } health_report { overall_health: HEALTHY } } }
I20260812 07:08:01.030081 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:01.024956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52454 (local address 127.31.156.67:35485)
0812 07:08:01.025285 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:01.025291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:01.025821 (+ 530us) server_negotiation.cc:407] Connection header received
0812 07:08:01.026005 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:01.026009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:01.026064 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:01.026149 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:01.026525 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:01.027209 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:01.028291 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:01.028585 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:01.028711 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:01.029284 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:01.029389 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:01.029459 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:01.029756 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 07:08:01.029871 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":87,"threads_started":1}
I20260812 07:08:01.045377 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:01.038422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35485 (local address 127.31.156.67:58067)
0812 07:08:01.039486 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:01.039687 (+ 201us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:01.039845 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:01.040122 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:01.040125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:01.040132 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:08:01.040228 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:01.040232 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:01.041450 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:01.041453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:01.042176 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:01.042191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:01.043964 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:01.043968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:01.044049 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:01.044082 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:08:01.044916 (+ 834us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:08:01.044925 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 07:08:01.045021 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 07:08:01.045126 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":132,"threads_started":1}
I20260812 07:08:01.045401 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:01.038489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:58067 (local address 127.31.156.67:35485)
0812 07:08:01.038635 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:01.038677 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:01.039838 (+ 1161us) server_negotiation.cc:407] Connection header received
0812 07:08:01.039938 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:01.039942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:01.040002 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:01.040099 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:01.040345 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:01.041246 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:01.042325 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:01.043745 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:01.043880 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:01.044226 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:01.044681 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:01.044875 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:01.045120 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 07:08:01.045194 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:08:01.053272 2052 raft_consensus.cc:1261] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Refusing update from remote peer 3d9f5e32109f478f8493792225db338b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:01.053315 2219 raft_consensus.cc:1261] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Refusing update from remote peer 3d9f5e32109f478f8493792225db338b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:01.053834 2529 consensus_queue.cc:1035] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:01.054049 2490 consensus_queue.cc:1035] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [LEADER]: Connected to new peer: Peer: permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 07:08:01.073207 2526 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 07:08:02.651643 1876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:02.651851 1876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:02.654875 1876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76: Sending DeleteTablet for 3 replicas of tablet 99957fb0c9db47de82ec31a121d7fc7b
I20260812 07:08:02.655611 2023 tablet_service.cc:1514] Processing DeleteTablet for tablet 99957fb0c9db47de82ec31a121d7fc7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:02 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:36298
I20260812 07:08:02.655634 2356 tablet_service.cc:1514] Processing DeleteTablet for tablet 99957fb0c9db47de82ec31a121d7fc7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:02 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:52440
I20260812 07:08:02.655678 2196 tablet_service.cc:1514] Processing DeleteTablet for tablet 99957fb0c9db47de82ec31a121d7fc7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:02 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:60480
I20260812 07:08:02.656198 2573 tablet_replica.cc:330] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5: stopping tablet replica
I20260812 07:08:02.656376 2576 tablet_replica.cc:330] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b: stopping tablet replica
I20260812 07:08:02.656535 2573 raft_consensus.cc:2229] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:02.656605 2576 raft_consensus.cc:2229] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:02.656777 2573 raft_consensus.cc:2258] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:02.657003 2576 raft_consensus.cc:2258] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:02.657325 2575 tablet_replica.cc:330] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940: stopping tablet replica
I20260812 07:08:02.657531 2575 raft_consensus.cc:2229] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:02.657697 2575 raft_consensus.cc:2258] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:02.657750 2573 ts_tablet_manager.cc:1905] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:02.659281 2576 ts_tablet_manager.cc:1905] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:02.660439 2575 ts_tablet_manager.cc:1905] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:02.661412 2573 ts_tablet_manager.cc:1918] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:08:02.661578 2573 log.cc:1196] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/wal/wals/99957fb0c9db47de82ec31a121d7fc7b
I20260812 07:08:02.662005 2573 ts_tablet_manager.cc:1939] T 99957fb0c9db47de82ec31a121d7fc7b P c2302f3668514346910c50cadda3e7f5: Deleting consensus metadata
I20260812 07:08:02.662456 2576 ts_tablet_manager.cc:1918] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:08:02.662554 2576 log.cc:1196] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/99957fb0c9db47de82ec31a121d7fc7b
I20260812 07:08:02.662940 2576 ts_tablet_manager.cc:1939] T 99957fb0c9db47de82ec31a121d7fc7b P 3d9f5e32109f478f8493792225db338b: Deleting consensus metadata
I20260812 07:08:02.663579 1863 catalog_manager.cc:4841] TS c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199): tablet 99957fb0c9db47de82ec31a121d7fc7b (table test-table [id=643b9e8d11a042219fc26dd9af62f18b]) successfully deleted
I20260812 07:08:02.663734 2575 ts_tablet_manager.cc:1918] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:08:02.663820 2575 log.cc:1196] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/99957fb0c9db47de82ec31a121d7fc7b
I20260812 07:08:02.663801 1864 catalog_manager.cc:4841] TS 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485): tablet 99957fb0c9db47de82ec31a121d7fc7b (table test-table [id=643b9e8d11a042219fc26dd9af62f18b]) successfully deleted
I20260812 07:08:02.664170 2575 ts_tablet_manager.cc:1939] T 99957fb0c9db47de82ec31a121d7fc7b P 96eb3581fba04a3da45b174be1fa4940: Deleting consensus metadata
I20260812 07:08:02.665271 1864 catalog_manager.cc:4841] TS 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355): tablet 99957fb0c9db47de82ec31a121d7fc7b (table test-table [id=643b9e8d11a042219fc26dd9af62f18b]) successfully deleted
Aug 12 07:08:02 dist-test-slave-csg5 krb5kdc[1829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518482, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:08:02 dist-test-slave-csg5 krb5kdc[1829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518482, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:02.693486 2585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:02.682130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.31.156.126:41753)
0812 07:08:02.682484 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:02.682491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:02.682509 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:08:02.682749 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:02.682754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:02.682819 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:02.682924 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:02.683908 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:02.684640 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:02.685638 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:02.685923 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:02.686027 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:02.689289 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:02.689293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:02.689296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:02.689333 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:02.691396 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:02.692017 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:02.692020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:02.692021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:02.692115 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:02.692475 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:02.692478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:02.692488 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:02.692703 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:02.692821 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:02.693159 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 07:08:02.693319 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":180,"threads_started":1}
I20260812 07:08:02.697796 1876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:08:02.698287 1876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:02.704807 2196 tablet_service.cc:1467] Processing CreateTablet for tablet 107c475c3a8f4ccd8df34852f3da5c8f (DEFAULT_TABLE table=test-table [id=547a1975e1a34723aaefb3354b059a26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:02.704793 2023 tablet_service.cc:1467] Processing CreateTablet for tablet 107c475c3a8f4ccd8df34852f3da5c8f (DEFAULT_TABLE table=test-table [id=547a1975e1a34723aaefb3354b059a26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:02.705200 2023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107c475c3a8f4ccd8df34852f3da5c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:02.705201 2196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107c475c3a8f4ccd8df34852f3da5c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:02.705225 2356 tablet_service.cc:1467] Processing CreateTablet for tablet 107c475c3a8f4ccd8df34852f3da5c8f (DEFAULT_TABLE table=test-table [id=547a1975e1a34723aaefb3354b059a26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:02.705538 2356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107c475c3a8f4ccd8df34852f3da5c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:02.708232 2586 tablet_bootstrap.cc:492] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940: Bootstrap starting.
I20260812 07:08:02.708418 2587 tablet_bootstrap.cc:492] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5: Bootstrap starting.
I20260812 07:08:02.708721 2588 tablet_bootstrap.cc:492] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b: Bootstrap starting.
I20260812 07:08:02.709501 2586 tablet_bootstrap.cc:654] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:02.709664 2587 tablet_bootstrap.cc:654] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:02.709726 2588 tablet_bootstrap.cc:654] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:02.711078 2587 tablet_bootstrap.cc:492] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5: No bootstrap required, opened a new log
I20260812 07:08:02.711078 2588 tablet_bootstrap.cc:492] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b: No bootstrap required, opened a new log
I20260812 07:08:02.711186 2588 ts_tablet_manager.cc:1397] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:02.711107 2586 tablet_bootstrap.cc:492] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940: No bootstrap required, opened a new log
I20260812 07:08:02.711251 2587 ts_tablet_manager.cc:1397] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260812 07:08:02.711313 2586 ts_tablet_manager.cc:1397] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:02.711673 2588 raft_consensus.cc:348] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.711673 2587 raft_consensus.cc:348] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.711802 2588 raft_consensus.cc:374] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:02.711807 2587 raft_consensus.cc:374] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:02.711839 2588 raft_consensus.cc:729] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9f5e32109f478f8493792225db338b, State: Initialized, Role: FOLLOWER
I20260812 07:08:02.711843 2587 raft_consensus.cc:729] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2302f3668514346910c50cadda3e7f5, State: Initialized, Role: FOLLOWER
I20260812 07:08:02.711982 2588 consensus_queue.cc:260] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.711951 2586 raft_consensus.cc:348] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.712005 2587 consensus_queue.cc:260] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.712090 2586 raft_consensus.cc:374] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:02.712128 2586 raft_consensus.cc:729] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96eb3581fba04a3da45b174be1fa4940, State: Initialized, Role: FOLLOWER
I20260812 07:08:02.712297 2588 ts_tablet_manager.cc:1428] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:08:02.712388 2587 ts_tablet_manager.cc:1428] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 07:08:02.712291 2586 consensus_queue.cc:260] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.712769 2586 ts_tablet_manager.cc:1428] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 07:08:02.725528 2592 raft_consensus.cc:482] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:02.725708 2592 raft_consensus.cc:504] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.726140 2592 leader_election.cc:288] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355), 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485)
I20260812 07:08:02.726768 2217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c475c3a8f4ccd8df34852f3da5c8f" candidate_uuid: "c2302f3668514346910c50cadda3e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96eb3581fba04a3da45b174be1fa4940" is_pre_election: true
I20260812 07:08:02.726962 2217 raft_consensus.cc:2454] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2302f3668514346910c50cadda3e7f5 in term 0.
I20260812 07:08:02.727003 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c475c3a8f4ccd8df34852f3da5c8f" candidate_uuid: "c2302f3668514346910c50cadda3e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9f5e32109f478f8493792225db338b" is_pre_election: true
I20260812 07:08:02.727198 2380 raft_consensus.cc:2454] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2302f3668514346910c50cadda3e7f5 in term 0.
I20260812 07:08:02.727499 1981 leader_election.cc:302] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96eb3581fba04a3da45b174be1fa4940, c2302f3668514346910c50cadda3e7f5; no voters:
I20260812 07:08:02.727770 2592 raft_consensus.cc:2781] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:02.727870 2592 raft_consensus.cc:482] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:02.727911 2592 raft_consensus.cc:3037] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:02.729239 2592 raft_consensus.cc:504] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.729815 2592 leader_election.cc:288] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [CANDIDATE]: Term 1 election: Requested vote from peers 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355), 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485)
I20260812 07:08:02.730342 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c475c3a8f4ccd8df34852f3da5c8f" candidate_uuid: "c2302f3668514346910c50cadda3e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9f5e32109f478f8493792225db338b"
I20260812 07:08:02.730545 2380 raft_consensus.cc:3037] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:02.730508 2217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c475c3a8f4ccd8df34852f3da5c8f" candidate_uuid: "c2302f3668514346910c50cadda3e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96eb3581fba04a3da45b174be1fa4940"
I20260812 07:08:02.730814 2217 raft_consensus.cc:3037] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:02.731839 2380 raft_consensus.cc:2454] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2302f3668514346910c50cadda3e7f5 in term 1.
I20260812 07:08:02.732072 2217 raft_consensus.cc:2454] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2302f3668514346910c50cadda3e7f5 in term 1.
I20260812 07:08:02.732259 1981 leader_election.cc:302] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9f5e32109f478f8493792225db338b, c2302f3668514346910c50cadda3e7f5; no voters:
I20260812 07:08:02.732656 2592 raft_consensus.cc:2781] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:02.732959 2592 raft_consensus.cc:686] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 1 LEADER]: Becoming Leader. State: Replica: c2302f3668514346910c50cadda3e7f5, State: Running, Role: LEADER
I20260812 07:08:02.733158 2592 consensus_queue.cc:237] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.735277 1876 catalog_manager.cc:5495] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2302f3668514346910c50cadda3e7f5 (127.31.156.65). New cstate: current_term: 1 leader_uuid: "c2302f3668514346910c50cadda3e7f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } health_report { overall_health: UNKNOWN } } }
I20260812 07:08:02.759006 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:02.755166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.31.156.65:34199)
0812 07:08:02.755625 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:02.755633 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:02.755647 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:08:02.755698 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:02.755700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:02.755756 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:02.755930 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:02.756246 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:02.756806 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:02.757721 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:02.757959 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:02.758073 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:02.758401 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:02.758488 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:02.758577 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:02.758737 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 07:08:02.758806 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":211,"threads_started":1}
I20260812 07:08:02.768106 2217 raft_consensus.cc:1261] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Refusing update from remote peer c2302f3668514346910c50cadda3e7f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:02.768085 2380 raft_consensus.cc:1261] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Refusing update from remote peer c2302f3668514346910c50cadda3e7f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:02.768566 2592 consensus_queue.cc:1035] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:02.768890 2594 consensus_queue.cc:1035] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:02.811117 1874 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:02.811304 1874 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:02.813835 1874 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76: Sending DeleteTablet for 3 replicas of tablet 107c475c3a8f4ccd8df34852f3da5c8f
I20260812 07:08:02.814623 2023 tablet_service.cc:1514] Processing DeleteTablet for tablet 107c475c3a8f4ccd8df34852f3da5c8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:02 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:36298
I20260812 07:08:02.814771 2196 tablet_service.cc:1514] Processing DeleteTablet for tablet 107c475c3a8f4ccd8df34852f3da5c8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:02 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:60480
I20260812 07:08:02.814805 2356 tablet_service.cc:1514] Processing DeleteTablet for tablet 107c475c3a8f4ccd8df34852f3da5c8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:02 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:52440
I20260812 07:08:02.814962 2573 tablet_replica.cc:330] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5: stopping tablet replica
I20260812 07:08:02.815003 2576 tablet_replica.cc:330] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b: stopping tablet replica
I20260812 07:08:02.815116 2573 raft_consensus.cc:2229] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:02.815151 2576 raft_consensus.cc:2229] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:02.815289 2576 raft_consensus.cc:2258] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:02.815361 2573 raft_consensus.cc:2258] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:02.815641 2575 tablet_replica.cc:330] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940: stopping tablet replica
I20260812 07:08:02.815794 2575 raft_consensus.cc:2229] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:02.815966 2575 raft_consensus.cc:2258] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:02.816298 2573 ts_tablet_manager.cc:1905] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:02.816705 2576 ts_tablet_manager.cc:1905] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:02.816727 2575 ts_tablet_manager.cc:1905] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:02.819133 2576 ts_tablet_manager.cc:1918] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:08:02.819207 2576 log.cc:1196] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/107c475c3a8f4ccd8df34852f3da5c8f
I20260812 07:08:02.819453 2576 ts_tablet_manager.cc:1939] T 107c475c3a8f4ccd8df34852f3da5c8f P 3d9f5e32109f478f8493792225db338b: Deleting consensus metadata
I20260812 07:08:02.819628 2573 ts_tablet_manager.cc:1918] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:08:02.819717 2573 log.cc:1196] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-0/wal/wals/107c475c3a8f4ccd8df34852f3da5c8f
I20260812 07:08:02.819913 2575 ts_tablet_manager.cc:1918] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:08:02.819995 2575 log.cc:1196] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/107c475c3a8f4ccd8df34852f3da5c8f
I20260812 07:08:02.820061 2573 ts_tablet_manager.cc:1939] T 107c475c3a8f4ccd8df34852f3da5c8f P c2302f3668514346910c50cadda3e7f5: Deleting consensus metadata
I20260812 07:08:02.820302 2575 ts_tablet_manager.cc:1939] T 107c475c3a8f4ccd8df34852f3da5c8f P 96eb3581fba04a3da45b174be1fa4940: Deleting consensus metadata
I20260812 07:08:02.820557 1874 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 07:08:02.821233 1864 catalog_manager.cc:4841] TS 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355): tablet 107c475c3a8f4ccd8df34852f3da5c8f (table test-table [id=547a1975e1a34723aaefb3354b059a26]) successfully deleted
W20260812 07:08:02.821455 1874 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:02.821455 1864 catalog_manager.cc:4841] TS 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485): tablet 107c475c3a8f4ccd8df34852f3da5c8f (table test-table [id=547a1975e1a34723aaefb3354b059a26]) successfully deleted
I20260812 07:08:02.821483 1863 catalog_manager.cc:4841] TS c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199): tablet 107c475c3a8f4ccd8df34852f3da5c8f (table test-table [id=547a1975e1a34723aaefb3354b059a26]) successfully deleted
I20260812 07:08:02.828377 2023 tablet_service.cc:1467] Processing CreateTablet for tablet 6d78b057a7ce491f93e8854166e53deb (DEFAULT_TABLE table=test-table [id=70528a48a0cf49fc97dd7be0e38cbb68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:02.828367 2196 tablet_service.cc:1467] Processing CreateTablet for tablet 6d78b057a7ce491f93e8854166e53deb (DEFAULT_TABLE table=test-table [id=70528a48a0cf49fc97dd7be0e38cbb68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:02.828564 2356 tablet_service.cc:1467] Processing CreateTablet for tablet 6d78b057a7ce491f93e8854166e53deb (DEFAULT_TABLE table=test-table [id=70528a48a0cf49fc97dd7be0e38cbb68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:02.828781 2023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d78b057a7ce491f93e8854166e53deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:02.828781 2196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d78b057a7ce491f93e8854166e53deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:02.828985 2356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d78b057a7ce491f93e8854166e53deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:02.831642 2588 tablet_bootstrap.cc:492] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b: Bootstrap starting.
I20260812 07:08:02.831704 2587 tablet_bootstrap.cc:492] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5: Bootstrap starting.
I20260812 07:08:02.832657 2588 tablet_bootstrap.cc:654] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:02.832682 2587 tablet_bootstrap.cc:654] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:02.833092 2586 tablet_bootstrap.cc:492] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940: Bootstrap starting.
I20260812 07:08:02.834153 2588 tablet_bootstrap.cc:492] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b: No bootstrap required, opened a new log
I20260812 07:08:02.834198 2587 tablet_bootstrap.cc:492] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5: No bootstrap required, opened a new log
I20260812 07:08:02.834304 2587 ts_tablet_manager.cc:1397] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 07:08:02.834311 2588 ts_tablet_manager.cc:1397] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:02.834343 2586 tablet_bootstrap.cc:654] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:02.834836 2588 raft_consensus.cc:348] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.834982 2588 raft_consensus.cc:374] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:02.834892 2587 raft_consensus.cc:348] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.835029 2588 raft_consensus.cc:729] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9f5e32109f478f8493792225db338b, State: Initialized, Role: FOLLOWER
I20260812 07:08:02.835038 2587 raft_consensus.cc:374] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:02.835076 2587 raft_consensus.cc:729] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2302f3668514346910c50cadda3e7f5, State: Initialized, Role: FOLLOWER
I20260812 07:08:02.835182 2588 consensus_queue.cc:260] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.835239 2587 consensus_queue.cc:260] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.835542 2588 ts_tablet_manager.cc:1428] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 07:08:02.835562 2587 ts_tablet_manager.cc:1428] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:08:02.836405 2586 tablet_bootstrap.cc:492] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940: No bootstrap required, opened a new log
I20260812 07:08:02.836521 2586 ts_tablet_manager.cc:1397] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 07:08:02.837066 2586 raft_consensus.cc:348] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.837236 2586 raft_consensus.cc:374] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:02.837291 2586 raft_consensus.cc:729] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96eb3581fba04a3da45b174be1fa4940, State: Initialized, Role: FOLLOWER
I20260812 07:08:02.837455 2586 consensus_queue.cc:260] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.837960 2586 ts_tablet_manager.cc:1428] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260812 07:08:02.981683 2529 raft_consensus.cc:482] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:02.981863 2529 raft_consensus.cc:504] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.982571 2529 leader_election.cc:288] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199), 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355)
I20260812 07:08:02.983218 2052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d78b057a7ce491f93e8854166e53deb" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2302f3668514346910c50cadda3e7f5" is_pre_election: true
I20260812 07:08:02.983300 2217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d78b057a7ce491f93e8854166e53deb" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96eb3581fba04a3da45b174be1fa4940" is_pre_election: true
I20260812 07:08:02.983441 2052 raft_consensus.cc:2454] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d9f5e32109f478f8493792225db338b in term 0.
I20260812 07:08:02.983471 2217 raft_consensus.cc:2454] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d9f5e32109f478f8493792225db338b in term 0.
I20260812 07:08:02.983888 2305 leader_election.cc:302] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9f5e32109f478f8493792225db338b, c2302f3668514346910c50cadda3e7f5; no voters:
I20260812 07:08:02.984092 2529 raft_consensus.cc:2781] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:02.984283 2529 raft_consensus.cc:482] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:02.984324 2529 raft_consensus.cc:3037] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:02.985422 2529 raft_consensus.cc:504] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.985832 2529 leader_election.cc:288] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 election: Requested vote from peers c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199), 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355)
I20260812 07:08:02.986284 2052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d78b057a7ce491f93e8854166e53deb" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2302f3668514346910c50cadda3e7f5"
I20260812 07:08:02.986344 2217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d78b057a7ce491f93e8854166e53deb" candidate_uuid: "3d9f5e32109f478f8493792225db338b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96eb3581fba04a3da45b174be1fa4940"
I20260812 07:08:02.986428 2052 raft_consensus.cc:3037] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:02.986541 2217 raft_consensus.cc:3037] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:02.987568 2052 raft_consensus.cc:2454] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d9f5e32109f478f8493792225db338b in term 1.
I20260812 07:08:02.987943 2217 raft_consensus.cc:2454] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d9f5e32109f478f8493792225db338b in term 1.
I20260812 07:08:02.988026 2305 leader_election.cc:302] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9f5e32109f478f8493792225db338b, c2302f3668514346910c50cadda3e7f5; no voters:
I20260812 07:08:02.988412 2529 raft_consensus.cc:2781] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:02.988531 2529 raft_consensus.cc:686] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 1 LEADER]: Becoming Leader. State: Replica: 3d9f5e32109f478f8493792225db338b, State: Running, Role: LEADER
I20260812 07:08:02.988708 2529 consensus_queue.cc:237] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } }
I20260812 07:08:02.990741 1876 catalog_manager.cc:5495] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d9f5e32109f478f8493792225db338b (127.31.156.67). New cstate: current_term: 1 leader_uuid: "3d9f5e32109f478f8493792225db338b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d9f5e32109f478f8493792225db338b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35485 } health_report { overall_health: HEALTHY } } }
I20260812 07:08:03.042362 2608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:03.038828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.31.156.67:35485)
0812 07:08:03.039241 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:03.039248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:03.039260 (+ 12us) server_negotiation.cc:407] Connection header received
0812 07:08:03.039304 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:03.039307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:03.039350 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:03.039468 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:03.039749 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:03.040334 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:03.041180 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:03.041464 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:03.041587 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:03.041858 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:03.041942 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:03.042023 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:03.042120 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 07:08:03.042177 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":145,"threads_started":1}
I20260812 07:08:03.049433 2052 raft_consensus.cc:1261] T 6d78b057a7ce491f93e8854166e53deb P c2302f3668514346910c50cadda3e7f5 [term 1 FOLLOWER]: Refusing update from remote peer 3d9f5e32109f478f8493792225db338b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:03.049813 2217 raft_consensus.cc:1261] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Refusing update from remote peer 3d9f5e32109f478f8493792225db338b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:03.049837 2529 consensus_queue.cc:1035] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2302f3668514346910c50cadda3e7f5" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:03.050477 2529 consensus_queue.cc:1035] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [LEADER]: Connected to new peer: Peer: permanent_uuid: "96eb3581fba04a3da45b174be1fa4940" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 07:08:03.064852 2547 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 07:08:04.626973 1876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:04.627182 1876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:04.630236 1876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99ff1ff718fd44d5a854dae619519b76: Sending DeleteTablet for 3 replicas of tablet 6d78b057a7ce491f93e8854166e53deb
I20260812 07:08:04.631079 2196 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d78b057a7ce491f93e8854166e53deb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:04 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:60480
I20260812 07:08:04.631130 2023 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d78b057a7ce491f93e8854166e53deb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:04 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:36298
I20260812 07:08:04.631160 2356 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d78b057a7ce491f93e8854166e53deb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:04 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:52440
I20260812 07:08:04.631595 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 1957
I20260812 07:08:04.634511 2634 tablet_replica.cc:330] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940: stopping tablet replica
I20260812 07:08:04.634913 2634 raft_consensus.cc:2229] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:04.635128 2634 raft_consensus.cc:2258] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:04.635531 2635 tablet_replica.cc:330] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b: stopping tablet replica
I20260812 07:08:04.635694 2635 raft_consensus.cc:2229] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:04.635950 2635 raft_consensus.cc:2258] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:04.636129 2634 ts_tablet_manager.cc:1905] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:04.636575 2635 ts_tablet_manager.cc:1905] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:04.639211 2635 ts_tablet_manager.cc:1918] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:08:04.639335 2635 log.cc:1196] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/6d78b057a7ce491f93e8854166e53deb
I20260812 07:08:04.639690 2635 ts_tablet_manager.cc:1939] T 6d78b057a7ce491f93e8854166e53deb P 3d9f5e32109f478f8493792225db338b: Deleting consensus metadata
I20260812 07:08:04.640175 2634 ts_tablet_manager.cc:1918] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:08:04.640295 2634 log.cc:1196] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/6d78b057a7ce491f93e8854166e53deb
I20260812 07:08:04.640635 2634 ts_tablet_manager.cc:1939] T 6d78b057a7ce491f93e8854166e53deb P 96eb3581fba04a3da45b174be1fa4940: Deleting consensus metadata
I20260812 07:08:04.640880 1864 catalog_manager.cc:4841] TS 3d9f5e32109f478f8493792225db338b (127.31.156.67:35485): tablet 6d78b057a7ce491f93e8854166e53deb (table test-table [id=70528a48a0cf49fc97dd7be0e38cbb68]) successfully deleted
I20260812 07:08:04.642498 1864 catalog_manager.cc:4841] TS 96eb3581fba04a3da45b174be1fa4940 (127.31.156.66:43355): tablet 6d78b057a7ce491f93e8854166e53deb (table test-table [id=70528a48a0cf49fc97dd7be0e38cbb68]) successfully deleted
W20260812 07:08:04.647888 1863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:34199 (error 108)
W20260812 07:08:04.649482 2638 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:04.648947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34199 (local address 127.0.0.1:36328)
0812 07:08:04.649230 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:04.649330 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:34199: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":109,"threads_started":1}
W20260812 07:08:04.649649 1863 catalog_manager.cc:4584] TS c2302f3668514346910c50cadda3e7f5 (127.31.156.65:34199): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6d78b057a7ce491f93e8854166e53deb: Network error: Client connection negotiation failed: client connection to 127.31.156.65:34199: connect: Connection reset by peer (error 104)
I20260812 07:08:04.650231 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 2122
I20260812 07:08:04.661070 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 2285
I20260812 07:08:04.673219 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 1843
2026-08-12T07:08:04Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7201 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:04 dist-test-slave-csg5 krb5kdc[2641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:04 dist-test-slave-csg5 krb5kdc[2641](info): set up 2 sockets
Aug 12 07:08:04 dist-test-slave-csg5 krb5kdc[2641](info): commencing operation
krb5kdc: starting...
W20260812 07:08:05.884073 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:08:05 dist-test-slave-csg5 krb5kdc[2641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:08:06.043262 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:32893
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:32893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:06.177548 2655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:06.177915 2655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:06.177965 2655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:06.182992 2655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:06.183086 2655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:06.183107 2655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:06.183125 2655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:06.183140 2655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:06.188862 2655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:32893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:32893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:06.190009 2655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:06.191087 2655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:06.201508 2665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:06.201498 2663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:06.201710 2655 server_base.cc:1034] running on GCE node
W20260812 07:08:06.201502 2661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:06.202592 2655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:06.204068 2655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:06.205349 2655 hybrid_clock.cc:648] HybridClock initialized: now 1786518486205312 us; error 66 us; skew 500 ppm
Aug 12 07:08:06 dist-test-slave-csg5 krb5kdc[2641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518486, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:06.209002 2655 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:08:06.210026 2655 webserver.cc:460] Webserver started at http://127.31.156.126:40053/ using document root <none> and password file <none>
I20260812 07:08:06.210640 2655 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:06.210785 2655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:06.210980 2655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:06.213253 2655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "69b61b480b7a4a6c8b40cbe86ea7cbea"
format_stamp: "Formatted at 2026-08-12 07:08:06 on dist-test-slave-csg5"
server_key: "8f7e340de9e36c4fbf2ae22069f3c017"
server_key_iv: "76a12edd7dfd853be1640ced46b98685"
server_key_version: "encryptionkey@0"
I20260812 07:08:06.213835 2655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "69b61b480b7a4a6c8b40cbe86ea7cbea"
format_stamp: "Formatted at 2026-08-12 07:08:06 on dist-test-slave-csg5"
server_key: "8f7e340de9e36c4fbf2ae22069f3c017"
server_key_iv: "76a12edd7dfd853be1640ced46b98685"
server_key_version: "encryptionkey@0"
I20260812 07:08:06.218248 2655 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 07:08:06.221230 2672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:06.223062 2655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:06.223218 2655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "69b61b480b7a4a6c8b40cbe86ea7cbea"
format_stamp: "Formatted at 2026-08-12 07:08:06 on dist-test-slave-csg5"
server_key: "8f7e340de9e36c4fbf2ae22069f3c017"
server_key_iv: "76a12edd7dfd853be1640ced46b98685"
server_key_version: "encryptionkey@0"
I20260812 07:08:06.223333 2655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:06.244036 2655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:06.248409 2655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:06.248607 2655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:06.256364 2655 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:32893
I20260812 07:08:06.256489 2735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:32893 every 8 connection(s)
I20260812 07:08:06.257505 2655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:06.260788 2736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:06.262996 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 2655
I20260812 07:08:06.263140 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:06.263535 32369 external_mini_cluster.cc:1428] Setting key a5541e27c3c946659500c80a43d9ea3d
I20260812 07:08:06.268496 2736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea: Bootstrap starting.
I20260812 07:08:06.271095 2736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:06.272073 2736 log.cc:824] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:08:06 dist-test-slave-csg5 krb5kdc[2641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:06.274478 2736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea: No bootstrap required, opened a new log
I20260812 07:08:06.278532 2736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b61b480b7a4a6c8b40cbe86ea7cbea" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 32893 } }
I20260812 07:08:06.278859 2736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:06.278967 2736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b61b480b7a4a6c8b40cbe86ea7cbea, State: Initialized, Role: FOLLOWER
I20260812 07:08:06.279524 2736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b61b480b7a4a6c8b40cbe86ea7cbea" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 32893 } }
I20260812 07:08:06.279608 2739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:06.265169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59798 (local address 127.31.156.126:32893)
0812 07:08:06.265725 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:06.265747 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:06.265790 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:08:06.266559 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:06.266580 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:06.267118 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:06.267509 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:06.268434 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:06.269507 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:06.270435 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:06.271012 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:06.271201 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:06.273990 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:06.273997 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:06.274014 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:06.274043 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:06.276001 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:06.276561 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:06.276568 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:06.276573 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:06.276707 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:06.277036 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:06.277040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:06.277041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:06.277495 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:06.277728 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:06.278066 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 07:08:06.278429 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":155,"threads_started":1}
I20260812 07:08:06.279670 2736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:06.279750 2736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:06.279856 2736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:06.281011 2736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b61b480b7a4a6c8b40cbe86ea7cbea" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 32893 } }
I20260812 07:08:06.281415 2736 leader_election.cc:302] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69b61b480b7a4a6c8b40cbe86ea7cbea; no voters:
I20260812 07:08:06.281826 2736 leader_election.cc:288] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:06.281929 2741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:06.282217 2741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [term 1 LEADER]: Becoming Leader. State: Replica: 69b61b480b7a4a6c8b40cbe86ea7cbea, State: Running, Role: LEADER
I20260812 07:08:06.282624 2741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b61b480b7a4a6c8b40cbe86ea7cbea" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 32893 } }
I20260812 07:08:06.283377 2736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:06.284106 2741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69b61b480b7a4a6c8b40cbe86ea7cbea. Latest consensus state: current_term: 1 leader_uuid: "69b61b480b7a4a6c8b40cbe86ea7cbea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b61b480b7a4a6c8b40cbe86ea7cbea" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 32893 } } }
I20260812 07:08:06.284376 2741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:06.284672 2742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69b61b480b7a4a6c8b40cbe86ea7cbea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b61b480b7a4a6c8b40cbe86ea7cbea" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 32893 } } }
I20260812 07:08:06.284919 2742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:06.284945 2752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:06.288784 2752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:06.296818 2752 catalog_manager.cc:1347] Generated new cluster ID: f0265459bb504a50b7e1b4a27f9eb8ee
I20260812 07:08:06.296993 2752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:06.304819 2752 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:06.306339 2752 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:06.313079 2752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69b61b480b7a4a6c8b40cbe86ea7cbea: Generated new TSK 0
I20260812 07:08:06.314116 2752 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:08:06.358165 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:32893
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:06.492759 2764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:06.493065 2764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:06.493112 2764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:06.497504 2764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:06.497596 2764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:06.497661 2764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:06.504575 2764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:32893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:06.506052 2764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:06.507432 2764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:06.518250 2769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:06.518254 2772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:06.518250 2770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:06.518348 2764 server_base.cc:1034] running on GCE node
I20260812 07:08:06.519029 2764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:06.519733 2764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:06.520967 2764 hybrid_clock.cc:648] HybridClock initialized: now 1786518486520951 us; error 28 us; skew 500 ppm
Aug 12 07:08:06 dist-test-slave-csg5 krb5kdc[2641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518486, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:06.523952 2764 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:08:06.525059 2764 webserver.cc:460] Webserver started at http://127.31.156.65:41429/ using document root <none> and password file <none>
I20260812 07:08:06.525714 2764 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:06.525817 2764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:06.526072 2764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:06.528817 2764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "39f7d33d2dab4a73902bef589389c03a"
format_stamp: "Formatted at 2026-08-12 07:08:06 on dist-test-slave-csg5"
server_key: "3b545c97f8f44fb5f3ef51c8586d2aa3"
server_key_iv: "24fce3b084f8a4ab0f1eded49f42606a"
server_key_version: "encryptionkey@0"
I20260812 07:08:06.529476 2764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "39f7d33d2dab4a73902bef589389c03a"
format_stamp: "Formatted at 2026-08-12 07:08:06 on dist-test-slave-csg5"
server_key: "3b545c97f8f44fb5f3ef51c8586d2aa3"
server_key_iv: "24fce3b084f8a4ab0f1eded49f42606a"
server_key_version: "encryptionkey@0"
I20260812 07:08:06.533725 2764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 07:08:06.536660 2779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:06.538036 2764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:06.538203 2764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "39f7d33d2dab4a73902bef589389c03a"
format_stamp: "Formatted at 2026-08-12 07:08:06 on dist-test-slave-csg5"
server_key: "3b545c97f8f44fb5f3ef51c8586d2aa3"
server_key_iv: "24fce3b084f8a4ab0f1eded49f42606a"
server_key_version: "encryptionkey@0"
I20260812 07:08:06.538352 2764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:06.555603 2764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:06.559448 2764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:06.559707 2764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:06.560420 2764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:06.561355 2764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:06.561432 2764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:06.561506 2764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:06.561563 2764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:06.571959 2764 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:37657
I20260812 07:08:06.571985 2907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:37657 every 8 connection(s)
I20260812 07:08:06.572963 2764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:06.577481 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 2764
I20260812 07:08:06.577610 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:08:06.577935 32369 external_mini_cluster.cc:1428] Setting key 117e76bdd2de659fd9c57be272470089
Aug 12 07:08:06 dist-test-slave-csg5 krb5kdc[2641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518486, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:08:06.589895 2739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:06.575131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:45905 (local address 127.31.156.126:32893)
0812 07:08:06.575316 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:06.575321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:06.576319 (+ 998us) server_negotiation.cc:407] Connection header received
0812 07:08:06.577930 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:06.577933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:06.578012 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:06.578110 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:06.579643 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:06.580266 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:06.582104 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:06.582356 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:06.582473 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:06.585679 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:06.585683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:06.585685 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:06.585713 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:06.587433 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:06.588079 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:06.588083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:06.588085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:06.588283 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:06.588692 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:06.588696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:06.588698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:06.588899 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:06.589009 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:06.589573 (+ 564us) server_negotiation.cc:299] Negotiation successful
0812 07:08:06.589718 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:08:06.590946 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:06.575634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:32893 (local address 127.31.156.65:45905)
0812 07:08:06.576140 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:06.576186 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:06.577610 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:06.578315 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:06.578324 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:06.578837 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:06.579488 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:06.579505 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:06.580428 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:06.580434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:06.581962 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:06.581973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:06.582537 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:06.582541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:06.582786 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:06.583535 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:06.583554 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:06.585474 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:06.587584 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:06.587592 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:06.587611 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:06.587963 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:06.588394 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:06.588397 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:06.588399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:06.588581 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:06.589025 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:06.589035 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:06.589324 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 07:08:06.589532 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 07:08:06.589912 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":144,"threads_started":1}
I20260812 07:08:06.592562 2908 heartbeater.cc:347] Connected to a master server at 127.31.156.126:32893
I20260812 07:08:06.592882 2908 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
I20260812 07:08:06.593541 2908 heartbeater.cc:511] Master 127.31.156.126:32893 requested a full tablet report, sending...
I20260812 07:08:06.595464 2694 ts_manager.cc:194] Registered new tserver with Master: 39f7d33d2dab4a73902bef589389c03a (127.31.156.65:37657)
I20260812 07:08:06.596652 2694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:45905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:08:06.616345 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:32893
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:06.752622 2913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:06.752954 2913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:06.753005 2913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:06.757656 2913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:06.757740 2913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:06.757802 2913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:06.763415 2913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:32893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:06.764524 2913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:06.765559 2913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:06.776513 2922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:06.776613 2919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:06.776513 2918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:06.776731 2913 server_base.cc:1034] running on GCE node
I20260812 07:08:06.777403 2913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:06.778167 2913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:06.779433 2913 hybrid_clock.cc:648] HybridClock initialized: now 1786518486779395 us; error 55 us; skew 500 ppm
Aug 12 07:08:06 dist-test-slave-csg5 krb5kdc[2641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518486, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:06.782899 2913 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:08:06.783967 2913 webserver.cc:460] Webserver started at http://127.31.156.66:45743/ using document root <none> and password file <none>
I20260812 07:08:06.784641 2913 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:06.784732 2913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:06.785012 2913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:06.787634 2913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "d975f0148cf8443bb8baa0f4f18205ec"
format_stamp: "Formatted at 2026-08-12 07:08:06 on dist-test-slave-csg5"
server_key: "93789abeb661ce27e3a19a35074d9401"
server_key_iv: "d81afd3ceea35a0fbeeb2f081a67ca93"
server_key_version: "encryptionkey@0"
I20260812 07:08:06.788288 2913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "d975f0148cf8443bb8baa0f4f18205ec"
format_stamp: "Formatted at 2026-08-12 07:08:06 on dist-test-slave-csg5"
server_key: "93789abeb661ce27e3a19a35074d9401"
server_key_iv: "d81afd3ceea35a0fbeeb2f081a67ca93"
server_key_version: "encryptionkey@0"
I20260812 07:08:06.792944 2913 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:06.796061 2931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:06.797588 2913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:08:06.797737 2913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "d975f0148cf8443bb8baa0f4f18205ec"
format_stamp: "Formatted at 2026-08-12 07:08:06 on dist-test-slave-csg5"
server_key: "93789abeb661ce27e3a19a35074d9401"
server_key_iv: "d81afd3ceea35a0fbeeb2f081a67ca93"
server_key_version: "encryptionkey@0"
I20260812 07:08:06.797879 2913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:06.818621 2913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:06.822381 2913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:06.822623 2913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:06.823371 2913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:06.824321 2913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:06.824378 2913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:06.824453 2913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:06.824491 2913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:06.834477 2913 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:40823
I20260812 07:08:06.834504 3054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:40823 every 8 connection(s)
I20260812 07:08:06.835529 2913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:06.835891 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 2913
I20260812 07:08:06.836023 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:08:06.836328 32369 external_mini_cluster.cc:1428] Setting key b952b0949c4be40dc98bb01f2d67be2b
Aug 12 07:08:06 dist-test-slave-csg5 krb5kdc[2641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518486, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
I20260812 07:08:06.853410 2739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:06.837878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:51429 (local address 127.31.156.126:32893)
0812 07:08:06.838029 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:06.838034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:06.839938 (+ 1904us) server_negotiation.cc:407] Connection header received
0812 07:08:06.841173 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:06.841176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:06.841229 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:06.841306 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:06.842979 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:06.843857 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:06.845346 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:06.845783 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:06.845943 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:06.848822 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:06.848837 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:06.848843 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:06.848883 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:06.850546 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:06.851366 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:06.851370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:06.851372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:06.851465 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:06.852009 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:06.852013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:06.852014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:06.852223 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:06.852305 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:06.853042 (+ 737us) server_negotiation.cc:299] Negotiation successful
0812 07:08:06.853209 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:08:06.854328 3057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:06.838422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:32893 (local address 127.31.156.66:51429)
0812 07:08:06.839755 (+ 1333us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:06.839807 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:06.840899 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:06.841544 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:06.841557 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:06.842144 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:06.842818 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:06.842834 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:06.844095 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:06.844098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:06.845195 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:06.845206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:06.846000 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:06.846003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:06.846169 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:06.846911 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:06.846930 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:06.848599 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:06.850825 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:06.850832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:06.850852 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:06.851212 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:06.851591 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:06.851594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:06.851596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:06.851886 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:06.852355 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:06.852361 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:06.852705 (+ 344us) client_negotiation.cc:769] Sending connection context
0812 07:08:06.852959 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 07:08:06.853372 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"thread_start_us":887,"threads_started":1}
I20260812 07:08:06.855806 3055 heartbeater.cc:347] Connected to a master server at 127.31.156.126:32893
I20260812 07:08:06.856093 3055 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:06.856669 3055 heartbeater.cc:511] Master 127.31.156.126:32893 requested a full tablet report, sending...
I20260812 07:08:06.857834 2694 ts_manager.cc:194] Registered new tserver with Master: d975f0148cf8443bb8baa0f4f18205ec (127.31.156.66:40823)
I20260812 07:08:06.858454 2694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:51429
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:08:06.875972 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:32893
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:07.006413 3061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:07.006776 3061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:07.006827 3061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:07.011216 3061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:07.011302 3061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:07.011368 3061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:07.017169 3061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:32893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:07.018350 3061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:07.019439 3061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:07.030401 3067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:07.030376 3068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:07.030460 3070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:07.030601 3061 server_base.cc:1034] running on GCE node
I20260812 07:08:07.031145 3061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:07.031911 3061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:07.033123 3061 hybrid_clock.cc:648] HybridClock initialized: now 1786518487033073 us; error 64 us; skew 500 ppm
Aug 12 07:08:07 dist-test-slave-csg5 krb5kdc[2641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518487, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:07.037117 3061 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:08:07.038293 3061 webserver.cc:460] Webserver started at http://127.31.156.67:39903/ using document root <none> and password file <none>
I20260812 07:08:07.038997 3061 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:07.039108 3061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:07.039391 3061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:07.041766 3061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "9589df963b914bac9a9ba3ffe644200f"
format_stamp: "Formatted at 2026-08-12 07:08:07 on dist-test-slave-csg5"
server_key: "55bed5a948f09e76c5ba75669c2b6cea"
server_key_iv: "44581cf94ce7f4b8b3b0d78290e9bdce"
server_key_version: "encryptionkey@0"
I20260812 07:08:07.042490 3061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "9589df963b914bac9a9ba3ffe644200f"
format_stamp: "Formatted at 2026-08-12 07:08:07 on dist-test-slave-csg5"
server_key: "55bed5a948f09e76c5ba75669c2b6cea"
server_key_iv: "44581cf94ce7f4b8b3b0d78290e9bdce"
server_key_version: "encryptionkey@0"
I20260812 07:08:07.047290 3061 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:08:07.050385 3078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:07.051973 3061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:07.052163 3061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "9589df963b914bac9a9ba3ffe644200f"
format_stamp: "Formatted at 2026-08-12 07:08:07 on dist-test-slave-csg5"
server_key: "55bed5a948f09e76c5ba75669c2b6cea"
server_key_iv: "44581cf94ce7f4b8b3b0d78290e9bdce"
server_key_version: "encryptionkey@0"
I20260812 07:08:07.052366 3061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:07.071137 3061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:07.075573 3061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:07.075821 3061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:07.076503 3061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:07.077445 3061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:07.077524 3061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:07.077601 3061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:07.077653 3061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:07.089274 3061 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:46359
I20260812 07:08:07.089305 3208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:46359 every 8 connection(s)
I20260812 07:08:07.090339 3061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:07.095395 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 3061
I20260812 07:08:07.095537 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:08:07.095912 32369 external_mini_cluster.cc:1428] Setting key 7f94ff8362dab45cef905f4cb60146c0
Aug 12 07:08:07 dist-test-slave-csg5 krb5kdc[2641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518487, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:07.106143 2739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:07.092583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:39755 (local address 127.31.156.126:32893)
0812 07:08:07.092763 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:07.092769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:07.093835 (+ 1066us) server_negotiation.cc:407] Connection header received
0812 07:08:07.095420 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:07.095427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:07.095498 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:07.095604 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:07.096998 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:07.097545 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:07.098791 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:07.099034 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:07.099174 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:07.102087 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:07.102091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:07.102093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:07.102120 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:07.104036 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:07.104618 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:07.104621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:07.104622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:07.104708 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:07.105064 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:07.105066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:07.105067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:07.105279 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:07.105359 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:07.105894 (+ 535us) server_negotiation.cc:299] Negotiation successful
0812 07:08:07.106010 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:08:07.107173 3211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:07.093103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:32893 (local address 127.31.156.67:39755)
0812 07:08:07.093636 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:07.093698 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:07.095101 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:07.095824 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:07.095831 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:07.096294 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:07.096830 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:07.096844 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:07.097699 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:07.097704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:07.098624 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:07.098634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:07.099214 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:07.099218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:07.099433 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:07.100079 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:07.100098 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:07.101888 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:07.104174 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:07.104181 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:07.104196 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:07.104518 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:07.104804 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:07.104807 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:07.104808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:07.104970 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:07.105381 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:07.105387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:07.105661 (+ 274us) client_negotiation.cc:769] Sending connection context
0812 07:08:07.105829 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 07:08:07.106110 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":129,"threads_started":1}
I20260812 07:08:07.108662 3209 heartbeater.cc:347] Connected to a master server at 127.31.156.126:32893
I20260812 07:08:07.108992 3209 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:07.109629 3209 heartbeater.cc:511] Master 127.31.156.126:32893 requested a full tablet report, sending...
I20260812 07:08:07.110833 2694 ts_manager.cc:194] Registered new tserver with Master: 9589df963b914bac9a9ba3ffe644200f (127.31.156.67:46359)
I20260812 07:08:07.111374 2694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:39755
I20260812 07:08:07.122963 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 07:08:07.141279 2739 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:07.139308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.31.156.126:32893)
0812 07:08:07.139470 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:07.139473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:07.139709 (+ 236us) server_negotiation.cc:407] Connection header received
0812 07:08:07.139836 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:07.139839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:07.139889 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:07.139965 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:07.140980 (+ 1015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59804: BlockingRecv error: recv got EOF from 127.0.0.1:59804 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:08:07.143210 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 2764
I20260812 07:08:07.153903 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 2913
I20260812 07:08:07.163792 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 3061
I20260812 07:08:07.173887 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 2655
2026-08-12T07:08:07Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2496 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:07 dist-test-slave-csg5 krb5kdc[3222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:07 dist-test-slave-csg5 krb5kdc[3222](info): set up 2 sockets
Aug 12 07:08:07 dist-test-slave-csg5 krb5kdc[3222](info): commencing operation
krb5kdc: starting...
W20260812 07:08:08.410122 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:08:08 dist-test-slave-csg5 krb5kdc[3222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:08:08.571730 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:42691
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:42691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:08.702162 3235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:08.702561 3235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:08.702646 3235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:08.707197 3235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:08.707335 3235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:08.707373 3235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:08.707417 3235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:08.707458 3235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:08.714226 3235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:42691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:42691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:08.715549 3235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:08.716709 3235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:08.726372 3242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:08.726374 3246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:08.726423 3243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:08.726830 3235 server_base.cc:1034] running on GCE node
I20260812 07:08:08.727622 3235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:08.728857 3235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:08.730089 3235 hybrid_clock.cc:648] HybridClock initialized: now 1786518488730053 us; error 52 us; skew 500 ppm
Aug 12 07:08:08 dist-test-slave-csg5 krb5kdc[3222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518488, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:08.733618 3235 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:08:08.734907 3235 webserver.cc:460] Webserver started at http://127.31.156.126:36161/ using document root <none> and password file <none>
I20260812 07:08:08.735531 3235 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:08.735600 3235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:08.735844 3235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:08.738704 3235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "b58d41028ec949eaa0ab575c6db54e4a"
format_stamp: "Formatted at 2026-08-12 07:08:08 on dist-test-slave-csg5"
server_key: "ce740785b0d2212cd6bff876d8b8b536"
server_key_iv: "1a2119960eb3eca17be5b53ed94e37d2"
server_key_version: "encryptionkey@0"
I20260812 07:08:08.739387 3235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "b58d41028ec949eaa0ab575c6db54e4a"
format_stamp: "Formatted at 2026-08-12 07:08:08 on dist-test-slave-csg5"
server_key: "ce740785b0d2212cd6bff876d8b8b536"
server_key_iv: "1a2119960eb3eca17be5b53ed94e37d2"
server_key_version: "encryptionkey@0"
I20260812 07:08:08.743650 3235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:08:08.746567 3253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:08.748096 3235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:08:08.748291 3235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "b58d41028ec949eaa0ab575c6db54e4a"
format_stamp: "Formatted at 2026-08-12 07:08:08 on dist-test-slave-csg5"
server_key: "ce740785b0d2212cd6bff876d8b8b536"
server_key_iv: "1a2119960eb3eca17be5b53ed94e37d2"
server_key_version: "encryptionkey@0"
I20260812 07:08:08.748421 3235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:08.781903 3235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:08.785669 3235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:08.785923 3235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:08.794272 3235 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:42691
I20260812 07:08:08.794922 3316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:42691 every 8 connection(s)
I20260812 07:08:08.795641 3235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:08.798990 3317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:08.801254 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 3235
I20260812 07:08:08.801393 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:08.801680 32369 external_mini_cluster.cc:1428] Setting key e45e2daf9af80b06fc95d25cf2929f1c
I20260812 07:08:08.806428 3317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a: Bootstrap starting.
I20260812 07:08:08.808866 3317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:08.809737 3317 log.cc:824] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:08:08 dist-test-slave-csg5 krb5kdc[3222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:08.811926 3317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a: No bootstrap required, opened a new log
I20260812 07:08:08.815717 3317 raft_consensus.cc:348] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58d41028ec949eaa0ab575c6db54e4a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42691 } }
I20260812 07:08:08.816076 3317 raft_consensus.cc:374] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:08.816177 3317 raft_consensus.cc:729] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58d41028ec949eaa0ab575c6db54e4a, State: Initialized, Role: FOLLOWER
I20260812 07:08:08.816665 3317 consensus_queue.cc:260] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58d41028ec949eaa0ab575c6db54e4a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42691 } }
I20260812 07:08:08.816836 3317 raft_consensus.cc:388] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:08.816928 3317 raft_consensus.cc:482] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:08.816969 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:08.803175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55040 (local address 127.31.156.126:42691)
0812 07:08:08.803706 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:08.803722 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:08.803763 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:08:08.804530 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:08.804550 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:08.804921 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:08.805328 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:08.806222 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:08.807335 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:08.808251 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:08.808748 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:08.808929 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:08.811691 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:08.811697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:08.811712 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:08.811741 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:08.813677 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:08.814151 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:08.814165 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:08.814170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:08.814298 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:08.814596 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:08.814600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:08.814602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:08.815116 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:08.815366 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:08.815642 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 07:08:08.815950 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":133,"threads_started":1}
I20260812 07:08:08.817049 3317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:08.818331 3317 raft_consensus.cc:504] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58d41028ec949eaa0ab575c6db54e4a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42691 } }
I20260812 07:08:08.818765 3317 leader_election.cc:302] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b58d41028ec949eaa0ab575c6db54e4a; no voters:
I20260812 07:08:08.819180 3317 leader_election.cc:288] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:08.819296 3323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:08.819550 3323 raft_consensus.cc:686] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [term 1 LEADER]: Becoming Leader. State: Replica: b58d41028ec949eaa0ab575c6db54e4a, State: Running, Role: LEADER
I20260812 07:08:08.819985 3323 consensus_queue.cc:237] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58d41028ec949eaa0ab575c6db54e4a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42691 } }
I20260812 07:08:08.820385 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:08:08.820700 3317 sys_catalog.cc:564] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:08.821913 3324 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b58d41028ec949eaa0ab575c6db54e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58d41028ec949eaa0ab575c6db54e4a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42691 } } }
I20260812 07:08:08.822065 3324 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:08.822745 3334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:08.823393 3325 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b58d41028ec949eaa0ab575c6db54e4a. Latest consensus state: current_term: 1 leader_uuid: "b58d41028ec949eaa0ab575c6db54e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58d41028ec949eaa0ab575c6db54e4a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42691 } } }
I20260812 07:08:08.823508 3325 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:08.826834 3334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:08.833657 3334 catalog_manager.cc:1347] Generated new cluster ID: bc797693ccd04863b2ad229e3b2cb73d
I20260812 07:08:08.833848 3334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:08.844592 3334 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:08.845978 3334 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:08.853005 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:08.847039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55054 (local address 127.31.156.126:42691)
0812 07:08:08.847178 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:08.847183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:08.847505 (+ 322us) server_negotiation.cc:407] Connection header received
0812 07:08:08.847675 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:08.847681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:08.847742 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:08.847841 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:08.848425 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:08.849019 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:08.849924 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:08.850191 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:08.850314 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:08.851485 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:08.851490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:08.851493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:08.852104 (+ 611us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:08.852201 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:08.852230 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:08.852368 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:08.852731 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 07:08:08.852824 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:08:08.859951 3334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b58d41028ec949eaa0ab575c6db54e4a: Generated new TSK 0
I20260812 07:08:08.860805 3334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:08.865998 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 3235
2026-08-12T07:08:08Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1689 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:08 dist-test-slave-csg5 krb5kdc[3355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:08 dist-test-slave-csg5 krb5kdc[3355](info): set up 2 sockets
Aug 12 07:08:08 dist-test-slave-csg5 krb5kdc[3355](info): commencing operation
krb5kdc: starting...
W20260812 07:08:10.094861 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:08:10 dist-test-slave-csg5 krb5kdc[3355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:08:10.255640 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:36403
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:35563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:36403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:10.391831 3371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:10.392141 3371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:10.392186 3371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:10.399804 3371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:10.399925 3371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:10.399961 3371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:10.399991 3371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:10.400017 3371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:10.408270 3371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:35563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:36403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:36403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:10.409482 3371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:10.410564 3371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:10.421169 3382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:10.421121 3371 server_base.cc:1034] running on GCE node
W20260812 07:08:10.421056 3380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:10.421406 3379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:10.422101 3371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:10.423528 3371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:10.424723 3371 hybrid_clock.cc:648] HybridClock initialized: now 1786518490424668 us; error 73 us; skew 500 ppm
Aug 12 07:08:10 dist-test-slave-csg5 krb5kdc[3355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:10.434909 3371 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:08:10.436218 3371 webserver.cc:460] Webserver started at http://127.31.156.126:39679/ using document root <none> and password file <none>
I20260812 07:08:10.436923 3371 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:10.437001 3371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:10.437260 3371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:10.440012 3371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "5ccb990a5d1240a0bdf5b44bf2547bca"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-csg5"
server_key: "18105e4e4581f80d55715ddd67cac209"
server_key_iv: "788368c2a0388347641039235c857bcf"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.440702 3371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "5ccb990a5d1240a0bdf5b44bf2547bca"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-csg5"
server_key: "18105e4e4581f80d55715ddd67cac209"
server_key_iv: "788368c2a0388347641039235c857bcf"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.445493 3371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:10.448737 3393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:10.450336 3371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:10.450534 3371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "5ccb990a5d1240a0bdf5b44bf2547bca"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-csg5"
server_key: "18105e4e4581f80d55715ddd67cac209"
server_key_iv: "788368c2a0388347641039235c857bcf"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.450730 3371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:10.479898 3371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:10.483693 3371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:10.483935 3371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:10.492803 3371 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:36403
I20260812 07:08:10.493305 3454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:36403 every 8 connection(s)
I20260812 07:08:10.493966 3371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:10.495635 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 3371
I20260812 07:08:10.495787 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:10.496071 32369 external_mini_cluster.cc:1428] Setting key 323a74646fabd2277f5b77f74de0e823
I20260812 07:08:10.497758 3455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:10.505694 3455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca: Bootstrap starting.
Aug 12 07:08:10 dist-test-slave-csg5 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:10.509325 3455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:10.510425 3455 log.cc:824] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:10.513223 3455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca: No bootstrap required, opened a new log
I20260812 07:08:10.514137 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.497629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.31.156.126:36403)
0812 07:08:10.498078 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.498096 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.498134 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:08:10.498981 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.499002 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.499353 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.499637 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.500726 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.501775 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.503124 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.503672 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.503805 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.507291 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.507300 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.507319 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.507364 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.510023 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.510825 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.510836 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.510850 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.511087 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.511598 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.511602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.511603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.512105 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.512288 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.512746 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.513135 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":160,"threads_started":1}
I20260812 07:08:10.517416 3455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccb990a5d1240a0bdf5b44bf2547bca" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36403 } }
I20260812 07:08:10.517727 3455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:10.517829 3455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ccb990a5d1240a0bdf5b44bf2547bca, State: Initialized, Role: FOLLOWER
I20260812 07:08:10.518410 3455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccb990a5d1240a0bdf5b44bf2547bca" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36403 } }
I20260812 07:08:10.518602 3455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:10.518729 3455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:10.518879 3455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:10.520020 3455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccb990a5d1240a0bdf5b44bf2547bca" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36403 } }
I20260812 07:08:10.520469 3455 leader_election.cc:302] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ccb990a5d1240a0bdf5b44bf2547bca; no voters:
I20260812 07:08:10.520936 3455 leader_election.cc:288] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:10.521063 3460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:10.521359 3460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [term 1 LEADER]: Becoming Leader. State: Replica: 5ccb990a5d1240a0bdf5b44bf2547bca, State: Running, Role: LEADER
I20260812 07:08:10.521714 3460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccb990a5d1240a0bdf5b44bf2547bca" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36403 } }
I20260812 07:08:10.522397 3455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:10.523696 3463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ccb990a5d1240a0bdf5b44bf2547bca. Latest consensus state: current_term: 1 leader_uuid: "5ccb990a5d1240a0bdf5b44bf2547bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccb990a5d1240a0bdf5b44bf2547bca" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36403 } } }
I20260812 07:08:10.523851 3463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:10.524039 3462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ccb990a5d1240a0bdf5b44bf2547bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccb990a5d1240a0bdf5b44bf2547bca" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36403 } } }
I20260812 07:08:10.524231 3462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:10.524324 3471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:10.528081 3471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:10.535003 3471 catalog_manager.cc:1347] Generated new cluster ID: 0327b542d8b3446c841ce4b2500c9e30
I20260812 07:08:10.535104 3471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:10.546377 3471 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:10.547842 3471 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:10.557519 3471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ccb990a5d1240a0bdf5b44bf2547bca: Generated new TSK 0
I20260812 07:08:10.558537 3471 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:08:10.599956 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36403
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:35563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:10.734424 3486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:10.734774 3486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:10.734824 3486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:10.739101 3486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:10.739176 3486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:10.739238 3486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:10.744858 3486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:35563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:10.746110 3486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:10.747251 3486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:10.758234 3497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:10.758284 3493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:10.758234 3495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:10.758376 3486 server_base.cc:1034] running on GCE node
I20260812 07:08:10.759017 3486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:10.759790 3486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:10.761020 3486 hybrid_clock.cc:648] HybridClock initialized: now 1786518490760985 us; error 43 us; skew 500 ppm
Aug 12 07:08:10 dist-test-slave-csg5 krb5kdc[3355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:10.764566 3486 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:08:10.765776 3486 webserver.cc:460] Webserver started at http://127.31.156.65:34471/ using document root <none> and password file <none>
I20260812 07:08:10.766454 3486 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:10.766566 3486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:10.766875 3486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:10.769407 3486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "f83f181271aa4588badb7a1b14e6b374"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-csg5"
server_key: "e096302765b5c4ac4091b591112844b0"
server_key_iv: "b7db2ce5482fc28227984ec8c26b413f"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.770095 3486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "f83f181271aa4588badb7a1b14e6b374"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-csg5"
server_key: "e096302765b5c4ac4091b591112844b0"
server_key_iv: "b7db2ce5482fc28227984ec8c26b413f"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.774537 3486 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:08:10.777526 3505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:10.779057 3486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:08:10.779237 3486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "f83f181271aa4588badb7a1b14e6b374"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-csg5"
server_key: "e096302765b5c4ac4091b591112844b0"
server_key_iv: "b7db2ce5482fc28227984ec8c26b413f"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.779376 3486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:10.801394 3486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:10.805544 3486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:10.805809 3486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:10.806541 3486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:10.807562 3486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:10.807654 3486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:10.807749 3486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:10.807807 3486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:10.818498 3486 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:38391
I20260812 07:08:10.818572 3631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:38391 every 8 connection(s)
I20260812 07:08:10.819782 3486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:10.829829 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 3486
I20260812 07:08:10.830056 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:08:10.830472 32369 external_mini_cluster.cc:1428] Setting key cabc1a0d4f9fee866abb9fbb3b026e9a
Aug 12 07:08:10 dist-test-slave-csg5 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:10.837157 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.822077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:51823 (local address 127.31.156.126:36403)
0812 07:08:10.822228 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.822233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.823677 (+ 1444us) server_negotiation.cc:407] Connection header received
0812 07:08:10.825281 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.825286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.825350 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.825476 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.826863 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.827422 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.828610 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.828870 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.828990 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.832162 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.832166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.832168 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.832201 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.833891 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.834847 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.834852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.834853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.834982 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.835540 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.835544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.835554 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.835836 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.835952 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.836719 (+ 767us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.836896 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:08:10.838042 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.822636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:36403 (local address 127.31.156.65:51823)
0812 07:08:10.823499 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.823563 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:10.825035 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:10.825689 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:10.825697 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:10.826121 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:10.826693 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.826718 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.827560 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.827564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.828479 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.828490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.829042 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.829047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.829221 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.829940 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:10.829958 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:10.831940 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:10.834047 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.834058 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.834083 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.834612 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.835101 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.835104 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.835106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.835399 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.835972 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:10.835980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:10.836344 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 07:08:10.836537 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 07:08:10.836945 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":139,"threads_started":1}
I20260812 07:08:10.839593 3632 heartbeater.cc:347] Connected to a master server at 127.31.156.126:36403
I20260812 07:08:10.839923 3632 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:10.840674 3632 heartbeater.cc:511] Master 127.31.156.126:36403 requested a full tablet report, sending...
I20260812 07:08:10.842466 3415 ts_manager.cc:194] Registered new tserver with Master: f83f181271aa4588badb7a1b14e6b374 (127.31.156.65:38391)
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:08:10.843880 3415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:51823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:08:10.867542 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36403
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:35563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:10.996410 3638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:10.996714 3638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:10.996762 3638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:11.001133 3638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:11.001215 3638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:11.001281 3638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:11.007035 3638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:35563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:11.008291 3638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:11.009406 3638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:11.020363 3644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:11.020363 3646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:11.020779 3643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:11.021198 3638 server_base.cc:1034] running on GCE node
I20260812 07:08:11.021795 3638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:11.022852 3638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:11.024091 3638 hybrid_clock.cc:648] HybridClock initialized: now 1786518491023979 us; error 126 us; skew 500 ppm
Aug 12 07:08:11 dist-test-slave-csg5 krb5kdc[3355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518491, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:11.028422 3638 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:08:11.029536 3638 webserver.cc:460] Webserver started at http://127.31.156.66:38571/ using document root <none> and password file <none>
I20260812 07:08:11.030203 3638 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:11.030278 3638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:11.030510 3638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:11.032785 3638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "4cebcfaf68a4457d804a8f8a97469550"
format_stamp: "Formatted at 2026-08-12 07:08:11 on dist-test-slave-csg5"
server_key: "f285a7e0fadd760ff3c51034b198cd40"
server_key_iv: "06181e2f4c8313e9688456146de07bfb"
server_key_version: "encryptionkey@0"
I20260812 07:08:11.033390 3638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "4cebcfaf68a4457d804a8f8a97469550"
format_stamp: "Formatted at 2026-08-12 07:08:11 on dist-test-slave-csg5"
server_key: "f285a7e0fadd760ff3c51034b198cd40"
server_key_iv: "06181e2f4c8313e9688456146de07bfb"
server_key_version: "encryptionkey@0"
I20260812 07:08:11.038359 3638 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:08:11.041474 3653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:11.043123 3638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:11.043310 3638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "4cebcfaf68a4457d804a8f8a97469550"
format_stamp: "Formatted at 2026-08-12 07:08:11 on dist-test-slave-csg5"
server_key: "f285a7e0fadd760ff3c51034b198cd40"
server_key_iv: "06181e2f4c8313e9688456146de07bfb"
server_key_version: "encryptionkey@0"
I20260812 07:08:11.043453 3638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:11.070778 3638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:11.075032 3638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:11.075277 3638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:11.076121 3638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:11.077224 3638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:11.077291 3638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:11.077364 3638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:11.077400 3638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:11.090229 3638 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:33719
I20260812 07:08:11.090248 3778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:33719 every 8 connection(s)
I20260812 07:08:11.091490 3638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:11.096709 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 3638
I20260812 07:08:11.096879 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:08:11.097273 32369 external_mini_cluster.cc:1428] Setting key d8af8dcad0f75c25d9ef3a1e9bb2e76a
Aug 12 07:08:11 dist-test-slave-csg5 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518491, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:11.110628 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:11.094232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:35021 (local address 127.31.156.126:36403)
0812 07:08:11.094448 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.094456 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.095662 (+ 1206us) server_negotiation.cc:407] Connection header received
0812 07:08:11.098218 (+ 2556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.098222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.098278 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.098378 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.100613 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:11.101491 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:11.102925 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:11.103198 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:11.103334 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:11.106332 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:11.106336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:11.106338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:11.106367 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:11.108108 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:11.108783 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:11.108787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:11.108789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:11.108879 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:11.109289 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:11.109293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:11.109295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:11.109517 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:11.109647 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:11.110264 (+ 617us) server_negotiation.cc:299] Negotiation successful
0812 07:08:11.110418 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:08:11.111797 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:11.094737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:36403 (local address 127.31.156.66:35021)
0812 07:08:11.095410 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:11.095490 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:11.097832 (+ 2342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:11.098629 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:11.098644 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:11.099481 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:11.100391 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:11.100418 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:11.101666 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:11.101670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:11.102781 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:11.102793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:11.103386 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:11.103391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:11.103587 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:11.104314 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:11.104334 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:11.106103 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:11.108284 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:11.108292 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:11.108317 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:11.108641 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:11.108989 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:11.108992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:11.108995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:11.109169 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:11.109652 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:11.109662 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:11.109972 (+ 310us) client_negotiation.cc:769] Sending connection context
0812 07:08:11.110197 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 07:08:11.110594 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":137,"threads_started":1}
I20260812 07:08:11.113425 3779 heartbeater.cc:347] Connected to a master server at 127.31.156.126:36403
I20260812 07:08:11.113864 3779 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:11.114773 3779 heartbeater.cc:511] Master 127.31.156.126:36403 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
I20260812 07:08:11.116343 3415 ts_manager.cc:194] Registered new tserver with Master: 4cebcfaf68a4457d804a8f8a97469550 (127.31.156.66:33719)
I20260812 07:08:11.117268 3415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:35021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:08:11.140937 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36403
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:35563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:11.267267 3784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:11.267654 3784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:11.267738 3784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:11.272082 3784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:11.272217 3784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:11.272318 3784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:11.278491 3784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:35563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:11.279908 3784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:11.281100 3784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:11.293171 3792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:11.293272 3790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:11.293233 3789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:11.293512 3784 server_base.cc:1034] running on GCE node
I20260812 07:08:11.294258 3784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:11.295140 3784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:11.296350 3784 hybrid_clock.cc:648] HybridClock initialized: now 1786518491296321 us; error 51 us; skew 500 ppm
Aug 12 07:08:11 dist-test-slave-csg5 krb5kdc[3355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518491, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:11.300009 3784 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:08:11.301009 3784 webserver.cc:460] Webserver started at http://127.31.156.67:37651/ using document root <none> and password file <none>
I20260812 07:08:11.301632 3784 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:11.301704 3784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:11.301898 3784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:11.304235 3784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "ad7d243fb5204cebb99817b6fc514cee"
format_stamp: "Formatted at 2026-08-12 07:08:11 on dist-test-slave-csg5"
server_key: "e63897cda075a16709a3674239e96762"
server_key_iv: "ba07f5d8e8b6e763bef39bd4138b9ad7"
server_key_version: "encryptionkey@0"
I20260812 07:08:11.304833 3784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "ad7d243fb5204cebb99817b6fc514cee"
format_stamp: "Formatted at 2026-08-12 07:08:11 on dist-test-slave-csg5"
server_key: "e63897cda075a16709a3674239e96762"
server_key_iv: "ba07f5d8e8b6e763bef39bd4138b9ad7"
server_key_version: "encryptionkey@0"
I20260812 07:08:11.309404 3784 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:11.312498 3799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:11.314209 3784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:08:11.314352 3784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "ad7d243fb5204cebb99817b6fc514cee"
format_stamp: "Formatted at 2026-08-12 07:08:11 on dist-test-slave-csg5"
server_key: "e63897cda075a16709a3674239e96762"
server_key_iv: "ba07f5d8e8b6e763bef39bd4138b9ad7"
server_key_version: "encryptionkey@0"
I20260812 07:08:11.314471 3784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:11.336292 3784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:11.340544 3784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:11.340742 3784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:11.341526 3784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:11.342538 3784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:11.342621 3784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:11.342732 3784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:11.342777 3784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:11.354287 3784 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:41549
I20260812 07:08:11.354326 3922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:41549 every 8 connection(s)
I20260812 07:08:11.355649 3784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:11.360183 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 3784
I20260812 07:08:11.360400 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:08:11.360770 32369 external_mini_cluster.cc:1428] Setting key cc12bde78a5f8b4d23894d6813c34d48
Aug 12 07:08:11 dist-test-slave-csg5 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518491, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:11.372367 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:11.357882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:57709 (local address 127.31.156.126:36403)
0812 07:08:11.358051 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.358055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.359111 (+ 1056us) server_negotiation.cc:407] Connection header received
0812 07:08:11.360831 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.360834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.360899 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.360987 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.362637 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:11.363240 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:11.364465 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:11.364695 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:11.364816 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:11.368047 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:11.368051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:11.368053 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:11.368082 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:11.369821 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:11.370508 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:11.370512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:11.370513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:11.370600 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:11.371104 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:11.371108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:11.371110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:11.371325 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:11.371442 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:11.372017 (+ 575us) server_negotiation.cc:299] Negotiation successful
0812 07:08:11.372163 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:08:11.373313 3925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:11.358288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:36403 (local address 127.31.156.67:57709)
0812 07:08:11.358929 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:11.358979 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:11.360579 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:11.361175 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:11.361182 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:11.361784 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:11.362481 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:11.362499 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:11.363385 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:11.363388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:11.364318 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:11.364328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:11.364869 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:11.364873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:11.365053 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:11.365746 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:11.365766 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:11.367811 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:11.369998 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:11.370007 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:11.370031 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:11.370365 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:11.370824 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:11.370832 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:11.370833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:11.370995 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:11.371444 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:11.371455 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:11.371757 (+ 302us) client_negotiation.cc:769] Sending connection context
0812 07:08:11.371974 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 07:08:11.372401 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":141,"threads_started":1}
I20260812 07:08:11.375099 3923 heartbeater.cc:347] Connected to a master server at 127.31.156.126:36403
I20260812 07:08:11.375527 3923 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:11.376137 3923 heartbeater.cc:511] Master 127.31.156.126:36403 requested a full tablet report, sending...
I20260812 07:08:11.377307 3415 ts_manager.cc:194] Registered new tserver with Master: ad7d243fb5204cebb99817b6fc514cee (127.31.156.67:41549)
I20260812 07:08:11.378014 3415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:57709
I20260812 07:08:11.388389 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 07:08:11 dist-test-slave-csg5 krb5kdc[3355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518491, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 07:08:11 dist-test-slave-csg5 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518491, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:11.421388 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:11.410052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49738 (local address 127.31.156.126:36403)
0812 07:08:11.410393 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.410399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.410550 (+ 151us) server_negotiation.cc:407] Connection header received
0812 07:08:11.410788 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.410792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.410840 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.410934 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.411741 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:11.412313 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:11.413207 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:11.413447 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:11.413570 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:11.416519 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:11.416528 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:11.416530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:11.416565 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:11.418449 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:11.419428 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:11.419433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:11.419434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:11.419540 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:11.419970 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:11.419974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:11.419975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:11.420409 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:11.420620 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:11.421014 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 07:08:11.421184 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
W20260812 07:08:11.421931 3415 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49738
I20260812 07:08:11.423518 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 3486
I20260812 07:08:11.435547 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 3638
I20260812 07:08:11.445369 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 3784
I20260812 07:08:11.455259 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 3371
2026-08-12T07:08:11Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2591 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:11 dist-test-slave-csg5 krb5kdc[3935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:11 dist-test-slave-csg5 krb5kdc[3935](info): set up 2 sockets
Aug 12 07:08:11 dist-test-slave-csg5 krb5kdc[3935](info): commencing operation
krb5kdc: starting...
W20260812 07:08:12.663172 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:08:12 dist-test-slave-csg5 krb5kdc[3935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:08:12.820137 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:40063
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:40063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:12.946291 3949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:12.946806 3949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:12.946950 3949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:12.951292 3949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:12.951421 3949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:12.951464 3949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:12.951506 3949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:12.951540 3949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:12.957289 3949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:40063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:40063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:12.958570 3949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:12.959761 3949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:12.969327 3955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:12.969417 3957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:12.969578 3949 server_base.cc:1034] running on GCE node
W20260812 07:08:12.969408 3954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:12.970348 3949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:12.971637 3949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:12.972864 3949 hybrid_clock.cc:648] HybridClock initialized: now 1786518492972833 us; error 38 us; skew 500 ppm
Aug 12 07:08:12 dist-test-slave-csg5 krb5kdc[3935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518492, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:12.976573 3949 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:08:12.977700 3949 webserver.cc:460] Webserver started at http://127.31.156.126:46581/ using document root <none> and password file <none>
I20260812 07:08:12.978345 3949 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:12.978448 3949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:12.978771 3949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:12.981062 3949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "66e18991937e4e049374d25d3178f503"
format_stamp: "Formatted at 2026-08-12 07:08:12 on dist-test-slave-csg5"
server_key: "e6404f368549c33a7d5cbbd67a27a0ad"
server_key_iv: "a7a1700f4f4ba416882fda3ea8ec3161"
server_key_version: "encryptionkey@0"
I20260812 07:08:12.981691 3949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "66e18991937e4e049374d25d3178f503"
format_stamp: "Formatted at 2026-08-12 07:08:12 on dist-test-slave-csg5"
server_key: "e6404f368549c33a7d5cbbd67a27a0ad"
server_key_iv: "a7a1700f4f4ba416882fda3ea8ec3161"
server_key_version: "encryptionkey@0"
I20260812 07:08:12.985949 3949 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:08:12.988926 3964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:12.990633 3949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:08:12.990880 3949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "66e18991937e4e049374d25d3178f503"
format_stamp: "Formatted at 2026-08-12 07:08:12 on dist-test-slave-csg5"
server_key: "e6404f368549c33a7d5cbbd67a27a0ad"
server_key_iv: "a7a1700f4f4ba416882fda3ea8ec3161"
server_key_version: "encryptionkey@0"
I20260812 07:08:12.991030 3949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:13.008127 3949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:13.012270 3949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:13.012518 3949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:13.022100 3949 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:40063
I20260812 07:08:13.022151 4024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:40063 every 8 connection(s)
I20260812 07:08:13.023504 3949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:13.027374 4025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:13.030010 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 3949
I20260812 07:08:13.030162 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:13.030454 32369 external_mini_cluster.cc:1428] Setting key cc6a651caf63e910577691fc500d8a87
I20260812 07:08:13.035586 4025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503: Bootstrap starting.
I20260812 07:08:13.038039 4025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:13.038978 4025 log.cc:824] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:08:13 dist-test-slave-csg5 krb5kdc[3935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:13.041385 4025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503: No bootstrap required, opened a new log
I20260812 07:08:13.046597 4025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e18991937e4e049374d25d3178f503" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40063 } }
I20260812 07:08:13.046947 4025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:13.047020 4025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e18991937e4e049374d25d3178f503, State: Initialized, Role: FOLLOWER
I20260812 07:08:13.047575 4025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e18991937e4e049374d25d3178f503" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40063 } }
I20260812 07:08:13.047766 4025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:13.047806 4028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:13.032245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46878 (local address 127.31.156.126:40063)
0812 07:08:13.032821 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:13.032843 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:13.032899 (+ 56us) server_negotiation.cc:407] Connection header received
0812 07:08:13.033685 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:13.033708 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:13.034104 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:13.034564 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:13.035561 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:13.036730 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:13.037717 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:13.038315 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:13.038517 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:13.041682 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:13.041689 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:13.041708 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:13.041752 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:13.044056 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:13.044644 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:13.044652 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:13.044659 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:13.044843 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:13.045186 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:13.045190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:13.045192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:13.045815 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:13.046045 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:13.046328 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 07:08:13.046704 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":177,"threads_started":1}
I20260812 07:08:13.047847 4025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:13.047941 4025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:13.049100 4025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e18991937e4e049374d25d3178f503" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40063 } }
I20260812 07:08:13.049499 4025 leader_election.cc:302] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66e18991937e4e049374d25d3178f503; no voters:
I20260812 07:08:13.049891 4025 leader_election.cc:288] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:13.050046 4030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:13.050377 4030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [term 1 LEADER]: Becoming Leader. State: Replica: 66e18991937e4e049374d25d3178f503, State: Running, Role: LEADER
I20260812 07:08:13.050894 4030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e18991937e4e049374d25d3178f503" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40063 } }
I20260812 07:08:13.051347 4025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:13.052762 4034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66e18991937e4e049374d25d3178f503" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e18991937e4e049374d25d3178f503" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40063 } } }
I20260812 07:08:13.052937 4034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:13.052886 4032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66e18991937e4e049374d25d3178f503. Latest consensus state: current_term: 1 leader_uuid: "66e18991937e4e049374d25d3178f503" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e18991937e4e049374d25d3178f503" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40063 } } }
I20260812 07:08:13.052995 4032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:13.053622 4043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:13.057348 4043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:13.063565 4043 catalog_manager.cc:1347] Generated new cluster ID: 491f8cc7816943abbfd09a96f9fe2ad8
I20260812 07:08:13.063660 4043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:13.084254 4043 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:13.085749 4043 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:13.094081 4043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66e18991937e4e049374d25d3178f503: Generated new TSK 0
I20260812 07:08:13.095211 4043 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:08:13.140034 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40063
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:13.268399 4057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:13.268760 4057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:13.268816 4057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:13.273289 4057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:13.273370 4057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:13.273434 4057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:13.279374 4057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:13.280596 4057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:13.281658 4057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:08:13.293334 4057 server_base.cc:1034] running on GCE node
W20260812 07:08:13.293416 4065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:13.293292 4064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:13.293578 4067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:13.294134 4057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:13.295077 4057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:13.296370 4057 hybrid_clock.cc:648] HybridClock initialized: now 1786518493296284 us; error 63 us; skew 500 ppm
Aug 12 07:08:13 dist-test-slave-csg5 krb5kdc[3935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518493, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:13.300019 4057 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:08:13.301086 4057 webserver.cc:460] Webserver started at http://127.31.156.65:45561/ using document root <none> and password file <none>
I20260812 07:08:13.301731 4057 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:13.301801 4057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:13.302037 4057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:13.304451 4057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "4c77ebc9a35d4dc69b008bd9292a84ea"
format_stamp: "Formatted at 2026-08-12 07:08:13 on dist-test-slave-csg5"
server_key: "7863e0da3fd0c4b58c87eddc6c07f342"
server_key_iv: "66c63c6674f3e5190a380a2f61c16371"
server_key_version: "encryptionkey@0"
I20260812 07:08:13.305094 4057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "4c77ebc9a35d4dc69b008bd9292a84ea"
format_stamp: "Formatted at 2026-08-12 07:08:13 on dist-test-slave-csg5"
server_key: "7863e0da3fd0c4b58c87eddc6c07f342"
server_key_iv: "66c63c6674f3e5190a380a2f61c16371"
server_key_version: "encryptionkey@0"
I20260812 07:08:13.309592 4057 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 07:08:13.312543 4074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:13.314304 4057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:13.314448 4057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "4c77ebc9a35d4dc69b008bd9292a84ea"
format_stamp: "Formatted at 2026-08-12 07:08:13 on dist-test-slave-csg5"
server_key: "7863e0da3fd0c4b58c87eddc6c07f342"
server_key_iv: "66c63c6674f3e5190a380a2f61c16371"
server_key_version: "encryptionkey@0"
I20260812 07:08:13.314564 4057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:13.331882 4057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:13.335520 4057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:13.335701 4057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:13.336305 4057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:13.337177 4057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:13.337229 4057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:13.337272 4057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:13.337288 4057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:13.347117 4057 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:36265
I20260812 07:08:13.347144 4209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:36265 every 8 connection(s)
I20260812 07:08:13.348323 4057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:13.348794 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 4057
I20260812 07:08:13.349027 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:08:13.349393 32369 external_mini_cluster.cc:1428] Setting key 5249caf015faee9fa6adc7f6462dd968
Aug 12 07:08:13 dist-test-slave-csg5 krb5kdc[3935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518493, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
I20260812 07:08:13.365640 4028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:13.350511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:45507 (local address 127.31.156.126:40063)
0812 07:08:13.350690 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:13.350697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:13.352005 (+ 1308us) server_negotiation.cc:407] Connection header received
0812 07:08:13.354308 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:13.354312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:13.354387 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:13.354498 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:13.355874 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:13.356455 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:13.357767 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:13.358015 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:13.358136 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:13.361046 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:13.361049 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:13.361052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:13.361089 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:13.362932 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:13.363527 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:13.363531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:13.363533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:13.363660 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:13.364054 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:13.364057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:13.364059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:13.364349 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:13.364466 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:13.365267 (+ 801us) server_negotiation.cc:299] Negotiation successful
0812 07:08:13.365432 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:08:13.366766 4212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:13.351171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40063 (local address 127.31.156.65:45507)
0812 07:08:13.351813 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:13.351890 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:13.354056 (+ 2166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:13.354771 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:13.354779 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:13.355185 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:13.355702 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:13.355716 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:13.356597 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:13.356601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:13.357621 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:13.357632 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:13.358189 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:13.358193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:13.358380 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:13.359124 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:13.359143 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:13.360821 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:13.363068 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:13.363075 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:13.363096 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:13.363406 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:13.363777 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:13.363780 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:13.363782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:13.363945 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:13.364467 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:13.364475 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:13.364820 (+ 345us) client_negotiation.cc:769] Sending connection context
0812 07:08:13.365036 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 07:08:13.365499 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":145,"threads_started":1}
I20260812 07:08:13.368482 4210 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40063
I20260812 07:08:13.368875 4210 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:13.369654 4210 heartbeater.cc:511] Master 127.31.156.126:40063 requested a full tablet report, sending...
I20260812 07:08:13.371599 3982 ts_manager.cc:194] Registered new tserver with Master: 4c77ebc9a35d4dc69b008bd9292a84ea (127.31.156.65:36265)
I20260812 07:08:13.372957 3982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:45507
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:08:13.388217 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40063
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:13.537565 4216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:13.537905 4216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:13.537957 4216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:13.543900 4216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:13.544005 4216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:13.544086 4216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:13.550642 4216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:13.552013 4216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:13.553434 4216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:13.566114 4221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:13.566349 4222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:13.566133 4224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:13.566741 4216 server_base.cc:1034] running on GCE node
I20260812 07:08:13.567306 4216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:13.568184 4216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:13.569376 4216 hybrid_clock.cc:648] HybridClock initialized: now 1786518493569322 us; error 66 us; skew 500 ppm
Aug 12 07:08:13 dist-test-slave-csg5 krb5kdc[3935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518493, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:13.572927 4216 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:08:13.574086 4216 webserver.cc:460] Webserver started at http://127.31.156.66:46751/ using document root <none> and password file <none>
I20260812 07:08:13.574766 4216 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:13.574868 4216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:13.575127 4216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:13.577534 4216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "44c26b8aa64e4c09aa40e0688daadbf6"
format_stamp: "Formatted at 2026-08-12 07:08:13 on dist-test-slave-csg5"
server_key: "57b10a504f36383fcde9d34da3a8e19c"
server_key_iv: "8e0e59810672b17cf4228029e13c78b7"
server_key_version: "encryptionkey@0"
I20260812 07:08:13.578162 4216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "44c26b8aa64e4c09aa40e0688daadbf6"
format_stamp: "Formatted at 2026-08-12 07:08:13 on dist-test-slave-csg5"
server_key: "57b10a504f36383fcde9d34da3a8e19c"
server_key_iv: "8e0e59810672b17cf4228029e13c78b7"
server_key_version: "encryptionkey@0"
I20260812 07:08:13.582708 4216 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:08:13.585888 4232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:13.587738 4216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:13.587935 4216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "44c26b8aa64e4c09aa40e0688daadbf6"
format_stamp: "Formatted at 2026-08-12 07:08:13 on dist-test-slave-csg5"
server_key: "57b10a504f36383fcde9d34da3a8e19c"
server_key_iv: "8e0e59810672b17cf4228029e13c78b7"
server_key_version: "encryptionkey@0"
I20260812 07:08:13.588099 4216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:13.602409 4216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:13.606472 4216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:13.606804 4216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:13.607571 4216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:13.608520 4216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:13.608597 4216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:13.608675 4216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:13.608728 4216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:13.620673 4216 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:36689
I20260812 07:08:13.620697 4366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:36689 every 8 connection(s)
I20260812 07:08:13.621762 4216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:13.628446 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 4216
I20260812 07:08:13.628571 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:08:13.628877 32369 external_mini_cluster.cc:1428] Setting key 7d9b207a651c1215e7c3f9678982cbb6
Aug 12 07:08:13 dist-test-slave-csg5 krb5kdc[3935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518493, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:13.638914 4028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:13.624035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:44065 (local address 127.31.156.126:40063)
0812 07:08:13.624216 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:13.624220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:13.625295 (+ 1075us) server_negotiation.cc:407] Connection header received
0812 07:08:13.626904 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:13.626908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:13.626972 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:13.627086 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:13.628491 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:13.629225 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:13.630859 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:13.631094 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:13.631212 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:13.634045 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:13.634048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:13.634051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:13.634077 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:13.636109 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:13.636885 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:13.636890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:13.636892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:13.636999 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:13.637452 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:13.637459 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:13.637462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:13.637767 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:13.637889 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:13.638511 (+ 622us) server_negotiation.cc:299] Negotiation successful
0812 07:08:13.638700 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 07:08:13.639987 4369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:13.624505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40063 (local address 127.31.156.66:44065)
0812 07:08:13.625115 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:13.625163 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:13.626633 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:13.627275 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:13.627282 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:13.627678 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:13.628288 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:13.628311 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:13.629380 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:13.629386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:13.630720 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:13.630738 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:13.631260 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:13.631265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:13.631461 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:13.632167 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:13.632184 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:13.633843 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:13.636316 (+ 2473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:13.636321 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:13.636342 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:13.636742 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:13.637108 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:13.637111 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:13.637112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:13.637338 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:13.637891 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:13.637896 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:13.638163 (+ 267us) client_negotiation.cc:769] Sending connection context
0812 07:08:13.638417 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 07:08:13.639005 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":152,"threads_started":1}
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:08:13.641476 4367 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40063
I20260812 07:08:13.641844 4367 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:13.642477 4367 heartbeater.cc:511] Master 127.31.156.126:40063 requested a full tablet report, sending...
I20260812 07:08:13.643870 3982 ts_manager.cc:194] Registered new tserver with Master: 44c26b8aa64e4c09aa40e0688daadbf6 (127.31.156.66:36689)
I20260812 07:08:13.644526 3982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:44065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
I20260812 07:08:14.376001 4210 heartbeater.cc:503] Master 127.31.156.126:40063 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
W20260812 07:08:14.426621 32369 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.31.156.67: real 0.796s user 0.005s sys 0.000s
I20260812 07:08:14.427356 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40063
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:14.556207 4372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:14.556564 4372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:14.556651 4372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:14.561110 4372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:14.561225 4372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:14.561333 4372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:14.567574 4372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:14.568859 4372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:14.570045 4372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:14.581619 4377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:14.581627 4380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:14.581622 4378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:14.581845 4372 server_base.cc:1034] running on GCE node
I20260812 07:08:14.582558 4372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:14.583469 4372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:14.584720 4372 hybrid_clock.cc:648] HybridClock initialized: now 1786518494584704 us; error 42 us; skew 500 ppm
Aug 12 07:08:14 dist-test-slave-csg5 krb5kdc[3935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:14.588060 4372 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:08:14.589118 4372 webserver.cc:460] Webserver started at http://127.31.156.67:44055/ using document root <none> and password file <none>
I20260812 07:08:14.589684 4372 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:14.589769 4372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:14.589967 4372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:14.592316 4372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "724fe59347924febaf70a18fe574607e"
format_stamp: "Formatted at 2026-08-12 07:08:14 on dist-test-slave-csg5"
server_key: "2bad4957342877288808a8aec29fcefb"
server_key_iv: "be35c0adc4c26739e81cb9b0207c274c"
server_key_version: "encryptionkey@0"
I20260812 07:08:14.592926 4372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "724fe59347924febaf70a18fe574607e"
format_stamp: "Formatted at 2026-08-12 07:08:14 on dist-test-slave-csg5"
server_key: "2bad4957342877288808a8aec29fcefb"
server_key_iv: "be35c0adc4c26739e81cb9b0207c274c"
server_key_version: "encryptionkey@0"
I20260812 07:08:14.597694 4372 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:08:14.600780 4387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:14.602639 4372 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:14.602888 4372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "724fe59347924febaf70a18fe574607e"
format_stamp: "Formatted at 2026-08-12 07:08:14 on dist-test-slave-csg5"
server_key: "2bad4957342877288808a8aec29fcefb"
server_key_iv: "be35c0adc4c26739e81cb9b0207c274c"
server_key_version: "encryptionkey@0"
I20260812 07:08:14.603014 4372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:14.628752 4372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:14.632534 4372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:14.632717 4372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:14.633327 4372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:14.634179 4372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:14.634228 4372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:14.634279 4372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:14.634295 4372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:14.645980 4372 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:44075
I20260812 07:08:14.646066 4522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:44075 every 8 connection(s)
I20260812 07:08:14.647125 4372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:14.647119 4367 heartbeater.cc:503] Master 127.31.156.126:40063 was elected leader, sending a full tablet report...
I20260812 07:08:14.657366 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 4372
I20260812 07:08:14.657522 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:08:14.658095 32369 external_mini_cluster.cc:1428] Setting key 0187637d1e025d02a2228284e8b5e4d1
Aug 12 07:08:14 dist-test-slave-csg5 krb5kdc[3935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:14.665395 4526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.649308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:48469 (local address 127.31.156.126:40063)
0812 07:08:14.649619 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:14.649623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:14.650486 (+ 863us) server_negotiation.cc:407] Connection header received
0812 07:08:14.652370 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:14.652374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:14.652443 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:14.652597 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:14.654133 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.654847 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.656223 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.656501 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.656636 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:14.660470 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:14.660475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:14.660478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:14.660510 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:14.662473 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:14.663237 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:14.663241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:14.663242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:14.663372 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:14.663830 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:14.663833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:14.663835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:14.664064 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:14.664152 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:14.664987 (+ 835us) server_negotiation.cc:299] Negotiation successful
0812 07:08:14.665230 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":139,"threads_started":1}
I20260812 07:08:14.666522 4527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.649768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40063 (local address 127.31.156.67:48469)
0812 07:08:14.650311 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:14.650366 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:14.652116 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:14.652838 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:14.652850 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:14.653370 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:14.653951 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:14.653968 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.655015 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.655018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:14.656057 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:14.656068 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.656695 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.656699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:14.656908 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:14.657838 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:14.657858 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:14.660236 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:14.662623 (+ 2387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:14.662629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:14.662645 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:14.663017 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:14.663505 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:14.663508 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:14.663509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:14.663695 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:14.664198 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:14.664204 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:14.664616 (+ 412us) client_negotiation.cc:769] Sending connection context
0812 07:08:14.664841 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 07:08:14.665202 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":134,"threads_started":1}
I20260812 07:08:14.668275 4524 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40063
I20260812 07:08:14.668613 4524 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:14.669245 4524 heartbeater.cc:511] Master 127.31.156.126:40063 requested a full tablet report, sending...
I20260812 07:08:14.670640 3982 ts_manager.cc:194] Registered new tserver with Master: 724fe59347924febaf70a18fe574607e (127.31.156.67:44075)
I20260812 07:08:14.671455 3982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:48469
I20260812 07:08:14.673172 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 07:08:14 dist-test-slave-csg5 krb5kdc[3935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:08:14 dist-test-slave-csg5 krb5kdc[3935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:08:14.736580 4532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.722901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46762 (local address 127.31.156.65:36265)
0812 07:08:14.723332 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:14.723344 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:14.723384 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:08:14.723526 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:14.723538 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:14.723955 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:14.724175 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:14.724998 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.726555 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.727584 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.728692 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.728849 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:14.732047 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:14.732065 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:14.732093 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:14.732132 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:14.734301 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:14.734951 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:14.734957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:14.734959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:14.735087 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:14.735430 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:14.735432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:14.735434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:14.735726 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:14.736144 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:14.736253 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 07:08:14.736408 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":110,"threads_started":1}
Aug 12 07:08:14 dist-test-slave-csg5 krb5kdc[3935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:14.751536 4526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.740419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46888 (local address 127.31.156.126:40063)
0812 07:08:14.740589 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:14.740594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:14.740775 (+ 181us) server_negotiation.cc:407] Connection header received
0812 07:08:14.740862 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:14.740864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:14.740912 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:14.740992 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:14.741859 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.742442 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.743502 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.743743 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.743837 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:14.746911 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:14.746915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:14.746917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:14.746952 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:14.748826 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:14.749521 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:14.749525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:14.749527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:14.749646 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:14.750190 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:14.750193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:14.750195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:14.750492 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:14.750626 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:14.751128 (+ 502us) server_negotiation.cc:299] Negotiation successful
0812 07:08:14.751345 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 07:08:14.752050 3982 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46888
I20260812 07:08:14.752969 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 4057
I20260812 07:08:14.764979 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 4216
I20260812 07:08:14.774842 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 4372
I20260812 07:08:14.785248 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 3949
2026-08-12T07:08:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3330 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:15 dist-test-slave-csg5 krb5kdc[4537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:15 dist-test-slave-csg5 krb5kdc[4537](info): set up 2 sockets
Aug 12 07:08:15 dist-test-slave-csg5 krb5kdc[4537](info): commencing operation
krb5kdc: starting...
W20260812 07:08:16.412406 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.599s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:08:16 dist-test-slave-csg5 krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:08:16.572103 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:37441
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:34557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:37441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:16.701200 4552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:16.701531 4552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:16.701579 4552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:16.706538 4552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:16.706823 4552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:16.706908 4552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:16.706952 4552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:16.706990 4552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:16.713860 4552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:34557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:37441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:37441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:16.715215 4552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:16.716470 4552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:16.725853 4560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:16.725853 4558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:16.725939 4557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:16.726090 4552 server_base.cc:1034] running on GCE node
I20260812 07:08:16.726933 4552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:16.728125 4552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:16.729332 4552 hybrid_clock.cc:648] HybridClock initialized: now 1786518496729309 us; error 40 us; skew 500 ppm
Aug 12 07:08:16 dist-test-slave-csg5 krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518496, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:16.738801 4552 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:08:16.739893 4552 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:16.740032 4552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:16.740272 4552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:16.742602 4552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "41427893a36642c1bafe7b60887cec19"
format_stamp: "Formatted at 2026-08-12 07:08:16 on dist-test-slave-csg5"
server_key: "8533199ddde11102b2c0e59c7170078e"
server_key_iv: "92b34118ecf00393094da038a770a3d3"
server_key_version: "encryptionkey@0"
I20260812 07:08:16.743314 4552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "41427893a36642c1bafe7b60887cec19"
format_stamp: "Formatted at 2026-08-12 07:08:16 on dist-test-slave-csg5"
server_key: "8533199ddde11102b2c0e59c7170078e"
server_key_iv: "92b34118ecf00393094da038a770a3d3"
server_key_version: "encryptionkey@0"
I20260812 07:08:16.747666 4552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:08:16.750407 4568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:16.752100 4552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:16.752373 4552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "41427893a36642c1bafe7b60887cec19"
format_stamp: "Formatted at 2026-08-12 07:08:16 on dist-test-slave-csg5"
server_key: "8533199ddde11102b2c0e59c7170078e"
server_key_iv: "92b34118ecf00393094da038a770a3d3"
server_key_version: "encryptionkey@0"
I20260812 07:08:16.752516 4552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:16.768437 4552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:16.772141 4552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:16.772387 4552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:16.780411 4552 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:37441
I20260812 07:08:16.780828 4627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:37441 every 8 connection(s)
I20260812 07:08:16.781258 4552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:16.784513 4628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:16.791052 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 4552
I20260812 07:08:16.791153 4628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19: Bootstrap starting.
I20260812 07:08:16.791222 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:16.791513 32369 external_mini_cluster.cc:1428] Setting key af1933b7f7cb3b2898eacfb65b5a2da4
I20260812 07:08:16.794217 4628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:16.795282 4628 log.cc:824] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:16.797832 4628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19: No bootstrap required, opened a new log
Aug 12 07:08:16 dist-test-slave-csg5 krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:16.803166 4628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41427893a36642c1bafe7b60887cec19" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37441 } }
I20260812 07:08:16.803462 4628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:16.803581 4628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41427893a36642c1bafe7b60887cec19, State: Initialized, Role: FOLLOWER
I20260812 07:08:16.804136 4628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41427893a36642c1bafe7b60887cec19" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37441 } }
I20260812 07:08:16.804332 4628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:16.804440 4628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:16.804594 4628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:16.805672 4628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41427893a36642c1bafe7b60887cec19" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37441 } }
I20260812 07:08:16.806145 4628 leader_election.cc:302] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41427893a36642c1bafe7b60887cec19; no voters:
I20260812 07:08:16.806547 4628 leader_election.cc:288] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:16.806746 4635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:16.807040 4635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [term 1 LEADER]: Becoming Leader. State: Replica: 41427893a36642c1bafe7b60887cec19, State: Running, Role: LEADER
I20260812 07:08:16.807426 4635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41427893a36642c1bafe7b60887cec19" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37441 } }
I20260812 07:08:16.808167 4628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:16.809235 4637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41427893a36642c1bafe7b60887cec19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41427893a36642c1bafe7b60887cec19" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37441 } } }
I20260812 07:08:16.809258 4631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:16.793043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51702 (local address 127.31.156.126:37441)
0812 07:08:16.793494 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.793511 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.793548 (+ 37us) server_negotiation.cc:407] Connection header received
0812 07:08:16.794248 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.794267 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.794761 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.795071 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.796158 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.797238 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.798523 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.799116 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.799247 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.802495 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:16.802502 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:16.802518 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:16.802550 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:16.804928 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:16.805443 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:16.805449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:16.805455 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:16.805580 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:16.805927 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:16.805931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:16.805934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:16.806575 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:16.806848 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.807742 (+ 894us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.808088 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":152,"threads_started":1}
I20260812 07:08:16.809381 4637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:16.809623 4637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41427893a36642c1bafe7b60887cec19. Latest consensus state: current_term: 1 leader_uuid: "41427893a36642c1bafe7b60887cec19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41427893a36642c1bafe7b60887cec19" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37441 } } }
I20260812 07:08:16.809717 4637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:16.810324 4645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:16.813922 4645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:16.820498 4645 catalog_manager.cc:1347] Generated new cluster ID: d0a4801cb0c94267bbf7655d9c6da662
I20260812 07:08:16.820613 4645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:16.840301 4645 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:16.841563 4645 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:16.846196 4645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19: Generated new TSK 0
I20260812 07:08:16.847059 4645 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:08:16.891623 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:37441
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:34557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:17.025676 4662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:17.026006 4662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:17.026059 4662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:17.030561 4662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:17.030637 4662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:17.030758 4662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:17.036582 4662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:34557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:37441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:17.037827 4662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:17.039005 4662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:17.050185 4667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:17.050333 4671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:17.050185 4668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:17.050397 4662 server_base.cc:1034] running on GCE node
I20260812 07:08:17.051148 4662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:17.051951 4662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:17.053268 4662 hybrid_clock.cc:648] HybridClock initialized: now 1786518497053246 us; error 32 us; skew 500 ppm
Aug 12 07:08:17 dist-test-slave-csg5 krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518497, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:17.056780 4662 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:08:17.057756 4662 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:17.057869 4662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:17.058131 4662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:17.060592 4662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "5e95f524d0c64e909978554f15374d4d"
format_stamp: "Formatted at 2026-08-12 07:08:17 on dist-test-slave-csg5"
server_key: "6f1344d43abe8559a50cc7b9e72ab17f"
server_key_iv: "d5cce98d37290e83bc36b38bd27af995"
server_key_version: "encryptionkey@0"
I20260812 07:08:17.061223 4662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "5e95f524d0c64e909978554f15374d4d"
format_stamp: "Formatted at 2026-08-12 07:08:17 on dist-test-slave-csg5"
server_key: "6f1344d43abe8559a50cc7b9e72ab17f"
server_key_iv: "d5cce98d37290e83bc36b38bd27af995"
server_key_version: "encryptionkey@0"
I20260812 07:08:17.065747 4662 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 07:08:17.069223 4677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:17.070869 4662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:17.071023 4662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "5e95f524d0c64e909978554f15374d4d"
format_stamp: "Formatted at 2026-08-12 07:08:17 on dist-test-slave-csg5"
server_key: "6f1344d43abe8559a50cc7b9e72ab17f"
server_key_iv: "d5cce98d37290e83bc36b38bd27af995"
server_key_version: "encryptionkey@0"
I20260812 07:08:17.071161 4662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:17.099192 4662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:17.103097 4662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:17.103359 4662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:17.104040 4662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:17.104996 4662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:17.105074 4662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:17.105163 4662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:17.105216 4662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:17.115643 4662 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:42707
I20260812 07:08:17.115754 4815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:42707 every 8 connection(s)
I20260812 07:08:17.116401 4662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:17.121673 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 4662
I20260812 07:08:17.121840 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:08:17.122146 32369 external_mini_cluster.cc:1428] Setting key 45396efe1094af738f26ed93cd009b55
Aug 12 07:08:17 dist-test-slave-csg5 krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518497, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:17.132643 4631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.118532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:60129 (local address 127.31.156.126:37441)
0812 07:08:17.118734 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.118739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.119621 (+ 882us) server_negotiation.cc:407] Connection header received
0812 07:08:17.121118 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.121122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.121189 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.121282 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:17.122882 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.123524 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.125068 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.125512 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.125649 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.128588 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:17.128592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:17.128594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:17.128621 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:17.130304 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:17.130977 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:17.130985 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:17.130986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:17.131066 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:17.131441 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:17.131445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:17.131446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:17.131647 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:17.131758 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.132300 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.132454 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:08:17.133556 4818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.118888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:37441 (local address 127.31.156.65:60129)
0812 07:08:17.119452 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:17.119502 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:17.120870 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:17.121498 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:17.121506 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:17.121985 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:17.122629 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.122686 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.123750 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.123753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.124919 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.124930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.125702 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.125705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.125894 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.126620 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:17.126639 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:17.128378 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:17.130446 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:17.130453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:17.130479 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:17.130858 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:17.131165 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:17.131168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:17.131170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:17.131331 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:17.131758 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:17.131766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:17.132046 (+ 280us) client_negotiation.cc:769] Sending connection context
0812 07:08:17.132242 (+ 196us) client_negotiation.cc:240] Negotiation successful
0812 07:08:17.132614 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":130,"threads_started":1}
I20260812 07:08:17.135113 4816 heartbeater.cc:347] Connected to a master server at 127.31.156.126:37441
I20260812 07:08:17.135390 4816 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:17.135912 4816 heartbeater.cc:511] Master 127.31.156.126:37441 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
I20260812 07:08:17.137845 4579 ts_manager.cc:194] Registered new tserver with Master: 5e95f524d0c64e909978554f15374d4d (127.31.156.65:42707)
I20260812 07:08:17.139709 4579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:60129
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:08:17.159240 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:37441
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:34557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:17.287892 4824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:17.288281 4824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:17.288369 4824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:17.293191 4824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:17.293314 4824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:17.293424 4824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:17.300264 4824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:34557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:37441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:17.301620 4824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:17.302872 4824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:17.313922 4833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:17.313997 4830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:17.313921 4831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:17.314293 4824 server_base.cc:1034] running on GCE node
I20260812 07:08:17.314893 4824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:17.315632 4824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:17.316856 4824 hybrid_clock.cc:648] HybridClock initialized: now 1786518497316818 us; error 53 us; skew 500 ppm
Aug 12 07:08:17 dist-test-slave-csg5 krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518497, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:17.320503 4824 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:08:17.321554 4824 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:17.321702 4824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:17.322003 4824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:17.324806 4824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "2dfe309fa4f74e4f8bf3b74254e13b93"
format_stamp: "Formatted at 2026-08-12 07:08:17 on dist-test-slave-csg5"
server_key: "4f3646452525f50fdd5d1b44b4b646a6"
server_key_iv: "c2e1bcce636714e84474a560a48d12a7"
server_key_version: "encryptionkey@0"
I20260812 07:08:17.325542 4824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "2dfe309fa4f74e4f8bf3b74254e13b93"
format_stamp: "Formatted at 2026-08-12 07:08:17 on dist-test-slave-csg5"
server_key: "4f3646452525f50fdd5d1b44b4b646a6"
server_key_iv: "c2e1bcce636714e84474a560a48d12a7"
server_key_version: "encryptionkey@0"
I20260812 07:08:17.330161 4824 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:08:17.333405 4839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:17.334977 4824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:17.335143 4824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "2dfe309fa4f74e4f8bf3b74254e13b93"
format_stamp: "Formatted at 2026-08-12 07:08:17 on dist-test-slave-csg5"
server_key: "4f3646452525f50fdd5d1b44b4b646a6"
server_key_iv: "c2e1bcce636714e84474a560a48d12a7"
server_key_version: "encryptionkey@0"
I20260812 07:08:17.335372 4824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:17.361387 4824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:17.365510 4824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:17.365787 4824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:17.366520 4824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:17.367560 4824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:17.367647 4824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:17.367735 4824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:17.367785 4824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:17.378875 4824 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:34219
I20260812 07:08:17.378904 4977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:34219 every 8 connection(s)
I20260812 07:08:17.379618 4824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:17.388538 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 4824
I20260812 07:08:17.388697 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:08:17.389024 32369 external_mini_cluster.cc:1428] Setting key 651c6c6f0f0fdf25f777316e9e9c6c8c
Aug 12 07:08:17 dist-test-slave-csg5 krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518497, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:17.396217 4631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.381768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:53335 (local address 127.31.156.126:37441)
0812 07:08:17.381940 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.381944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.382957 (+ 1013us) server_negotiation.cc:407] Connection header received
0812 07:08:17.384632 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.384636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.384718 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.384840 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:17.386257 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.386949 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.388300 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.388548 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.388677 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.391888 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:17.391892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:17.391894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:17.391921 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:17.393756 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:17.394406 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:17.394410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:17.394412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:17.394510 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:17.394949 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:17.394954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:17.394956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:17.395169 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:17.395286 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.395875 (+ 589us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.396023 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:08:17.397209 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.382122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:37441 (local address 127.31.156.66:53335)
0812 07:08:17.382780 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:17.382832 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:17.384363 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:17.385063 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:17.385071 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:17.385535 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:17.386078 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.386093 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.387103 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.387107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.388121 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.388131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.388733 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.388738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.388947 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.389783 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:17.389811 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:17.391687 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:17.393921 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:17.393928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:17.393946 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:17.394287 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:17.394615 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:17.394618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:17.394621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:17.394836 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:17.395288 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:17.395298 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:17.395588 (+ 290us) client_negotiation.cc:769] Sending connection context
0812 07:08:17.395818 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 07:08:17.396224 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":157,"threads_started":1}
I20260812 07:08:17.398731 4978 heartbeater.cc:347] Connected to a master server at 127.31.156.126:37441
I20260812 07:08:17.399065 4978 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:17.399775 4978 heartbeater.cc:511] Master 127.31.156.126:37441 requested a full tablet report, sending...
I20260812 07:08:17.401322 4579 ts_manager.cc:194] Registered new tserver with Master: 2dfe309fa4f74e4f8bf3b74254e13b93 (127.31.156.66:34219)
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:08:17.402320 4579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:53335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:08:17.428057 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:37441
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:34557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:17.549484 4983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:17.549795 4983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:17.549844 4983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:17.554136 4983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:17.554235 4983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:17.554307 4983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:17.560375 4983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:34557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:37441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:17.561573 4983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:17.562608 4983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:17.573158 4989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:17.573385 4983 server_base.cc:1034] running on GCE node
W20260812 07:08:17.573164 4991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:17.573148 4988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:17.574012 4983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:17.574862 4983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:17.576087 4983 hybrid_clock.cc:648] HybridClock initialized: now 1786518497576065 us; error 33 us; skew 500 ppm
Aug 12 07:08:17 dist-test-slave-csg5 krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518497, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:17.579335 4983 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:08:17.580130 4983 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:17.580199 4983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:17.580463 4983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:17.582635 4983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "42d8271920324faeb41b86b25273e068"
format_stamp: "Formatted at 2026-08-12 07:08:17 on dist-test-slave-csg5"
server_key: "aeee7b84ca806bde9dd59a607f633cf3"
server_key_iv: "3425ab9c4826277c21e1128ff54a1adf"
server_key_version: "encryptionkey@0"
I20260812 07:08:17.583324 4983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "42d8271920324faeb41b86b25273e068"
format_stamp: "Formatted at 2026-08-12 07:08:17 on dist-test-slave-csg5"
server_key: "aeee7b84ca806bde9dd59a607f633cf3"
server_key_iv: "3425ab9c4826277c21e1128ff54a1adf"
server_key_version: "encryptionkey@0"
I20260812 07:08:17.587517 4983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 07:08:17.590247 4997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:17.591779 4983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:17.591946 4983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "42d8271920324faeb41b86b25273e068"
format_stamp: "Formatted at 2026-08-12 07:08:17 on dist-test-slave-csg5"
server_key: "aeee7b84ca806bde9dd59a607f633cf3"
server_key_iv: "3425ab9c4826277c21e1128ff54a1adf"
server_key_version: "encryptionkey@0"
I20260812 07:08:17.592087 4983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:17.613212 4983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:17.616417 4983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:17.616652 4983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:17.617332 4983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:17.618271 4983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:17.618366 4983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:17.618450 4983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:17.618505 4983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:17.628842 4983 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:42877
I20260812 07:08:17.628870 5137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:42877 every 8 connection(s)
I20260812 07:08:17.629541 4983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:17.637010 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 4983
I20260812 07:08:17.637190 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:08:17.637547 32369 external_mini_cluster.cc:1428] Setting key 84c451aee0aa41f4b7ffb04a554916d9
Aug 12 07:08:17 dist-test-slave-csg5 krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518497, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:17.646569 4631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.631689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:37497 (local address 127.31.156.126:37441)
0812 07:08:17.631887 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.631891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.632836 (+ 945us) server_negotiation.cc:407] Connection header received
0812 07:08:17.634406 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.634410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.634470 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.634565 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:17.636084 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.636818 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.638570 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.638943 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.639075 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.642164 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:17.642168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:17.642170 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:17.642200 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:17.643970 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:17.644704 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:17.644708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:17.644710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:17.644812 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:17.645235 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:17.645238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:17.645240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:17.645464 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:17.645611 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.646220 (+ 609us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.646374 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 07:08:17.647655 5141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.632077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:37441 (local address 127.31.156.67:37497)
0812 07:08:17.632604 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:17.632654 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:17.634091 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:17.634901 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:17.634909 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:17.635345 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:17.635884 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.635900 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.636997 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.637000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.638429 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.638441 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.639172 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.639177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.639471 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.640252 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:17.640280 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:17.641959 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:17.644138 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:17.644146 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:17.644255 (+ 109us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:17.644571 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:17.644923 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:17.644927 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:17.644929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:17.645094 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:17.645610 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:17.645619 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:17.645935 (+ 316us) client_negotiation.cc:769] Sending connection context
0812 07:08:17.646178 (+ 243us) client_negotiation.cc:240] Negotiation successful
0812 07:08:17.646581 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":139,"threads_started":1}
I20260812 07:08:17.649114 5138 heartbeater.cc:347] Connected to a master server at 127.31.156.126:37441
I20260812 07:08:17.649457 5138 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:17.649994 5138 heartbeater.cc:511] Master 127.31.156.126:37441 requested a full tablet report, sending...
I20260812 07:08:17.651501 4579 ts_manager.cc:194] Registered new tserver with Master: 42d8271920324faeb41b86b25273e068 (127.31.156.67:42877)
I20260812 07:08:17.652446 4579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:37497
I20260812 07:08:17.652631 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:08:17.664024 4631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.655136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.31.156.126:37441)
0812 07:08:17.655373 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.655380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.655541 (+ 161us) server_negotiation.cc:407] Connection header received
0812 07:08:17.655592 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.655595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.655643 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.655723 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:17.656580 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.657310 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.658344 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.658568 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.658759 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.660399 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:17.660402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:17.660405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:17.660434 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:17.661942 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:17.662464 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:17.662468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:17.662469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:17.662569 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:17.662993 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:17.662996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:17.662997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:17.663273 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:17.663409 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.663679 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.663795 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260812 07:08:17.667585 4579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:08:17.671249 4579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:17.688759 5159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.681306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:42707 (local address 127.0.0.1:43328)
0812 07:08:17.681761 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:17.681838 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:17.681987 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:17.683469 (+ 1482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:17.683475 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:17.683498 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:17.683702 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.683710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.685269 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.685273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.687044 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.687056 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.688279 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.688284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.688388 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.688416 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 07:08:17.688476 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 07:08:17.688560 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":152,"thread_start_us":121,"threads_started":1}
I20260812 07:08:17.689399 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.682164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:34219 (local address 127.0.0.1:50560)
0812 07:08:17.682994 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:17.683009 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:17.683105 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:17.683711 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:17.683714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:17.683731 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:17.683839 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.683845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.685269 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.685281 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.687423 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.687434 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.688751 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.688754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.688866 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.689079 (+ 213us) client_negotiation.cc:769] Sending connection context
0812 07:08:17.689156 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 07:08:17.689246 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":71,"thread_start_us":113,"threads_started":1}
I20260812 07:08:17.689589 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.681455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43328 (local address 127.31.156.65:42707)
0812 07:08:17.682950 (+ 1495us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.682958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.682984 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:08:17.683076 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.683083 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.683316 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.683480 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:17.683832 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.685118 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.687192 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.688109 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.688863 (+ 754us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.689032 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.689091 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.689385 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1350,"thread_start_us":144,"threads_started":1}
I20260812 07:08:17.688766 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.681744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:42877 (local address 127.0.0.1:43074)
0812 07:08:17.682458 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:17.682473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:17.682567 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:17.683531 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:17.683535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:17.683554 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:17.683718 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.683725 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.685072 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.685076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.686820 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.686862 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.688030 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.688034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.688161 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.688207 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 07:08:17.688359 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 07:08:17.688523 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":82,"thread_start_us":118,"threads_started":1}
I20260812 07:08:17.689845 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.682273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43074 (local address 127.31.156.67:42877)
0812 07:08:17.682975 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.682988 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.683022 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:08:17.683130 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.683137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.683346 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.683891 (+ 545us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:17.684061 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.684873 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.686996 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.687872 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.688025 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.688224 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.688408 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.689659 (+ 1251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":124,"threads_started":1}
I20260812 07:08:17.690999 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.682439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.31.156.66:34219)
0812 07:08:17.682943 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.682956 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.683110 (+ 154us) server_negotiation.cc:407] Connection header received
0812 07:08:17.683203 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.683208 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.683367 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.683518 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:17.683966 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.685118 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.687584 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.688427 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.688527 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.688684 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.689676 (+ 992us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.690842 (+ 1166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":152,"threads_started":1}
I20260812 07:08:17.692698 4733 tablet_service.cc:1467] Processing CreateTablet for tablet 4e2ec2e8386d47a198d4156a5e82475b (DEFAULT_TABLE table=test-table [id=2527d42f55a74671bc38f1ee8f41410f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:17.692689 5057 tablet_service.cc:1467] Processing CreateTablet for tablet 4e2ec2e8386d47a198d4156a5e82475b (DEFAULT_TABLE table=test-table [id=2527d42f55a74671bc38f1ee8f41410f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:17.693858 4733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e2ec2e8386d47a198d4156a5e82475b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:17.693888 5057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e2ec2e8386d47a198d4156a5e82475b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:17.696647 4893 tablet_service.cc:1467] Processing CreateTablet for tablet 4e2ec2e8386d47a198d4156a5e82475b (DEFAULT_TABLE table=test-table [id=2527d42f55a74671bc38f1ee8f41410f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:17.697980 4893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e2ec2e8386d47a198d4156a5e82475b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:17.701339 5165 tablet_bootstrap.cc:492] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: Bootstrap starting.
I20260812 07:08:17.703744 5165 tablet_bootstrap.cc:654] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:17.704982 5165 log.cc:824] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:17.705598 5166 tablet_bootstrap.cc:492] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d: Bootstrap starting.
I20260812 07:08:17.707191 5169 tablet_bootstrap.cc:492] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: Bootstrap starting.
I20260812 07:08:17.707682 5165 tablet_bootstrap.cc:492] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: No bootstrap required, opened a new log
I20260812 07:08:17.707924 5165 ts_tablet_manager.cc:1397] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:08:17.708560 5166 tablet_bootstrap.cc:654] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:17.709637 5169 tablet_bootstrap.cc:654] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:17.709563 5166 log.cc:824] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:17.711083 5169 log.cc:824] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:17.712944 5165 raft_consensus.cc:348] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95f524d0c64e909978554f15374d4d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42707 } } peers { permanent_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34219 } } peers { permanent_uuid: "42d8271920324faeb41b86b25273e068" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42877 } }
I20260812 07:08:17.713297 5165 raft_consensus.cc:374] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:17.713348 5169 tablet_bootstrap.cc:492] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: No bootstrap required, opened a new log
I20260812 07:08:17.713361 5165 raft_consensus.cc:729] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42d8271920324faeb41b86b25273e068, State: Initialized, Role: FOLLOWER
I20260812 07:08:17.713371 5166 tablet_bootstrap.cc:492] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d: No bootstrap required, opened a new log
I20260812 07:08:17.713578 5169 ts_tablet_manager.cc:1397] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 07:08:17.713579 5166 ts_tablet_manager.cc:1397] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:08:17.714012 5165 consensus_queue.cc:260] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95f524d0c64e909978554f15374d4d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42707 } } peers { permanent_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34219 } } peers { permanent_uuid: "42d8271920324faeb41b86b25273e068" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42877 } }
I20260812 07:08:17.715214 5165 ts_tablet_manager.cc:1428] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 07:08:17.715698 5138 heartbeater.cc:503] Master 127.31.156.126:37441 was elected leader, sending a full tablet report...
I20260812 07:08:17.717559 5169 raft_consensus.cc:348] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95f524d0c64e909978554f15374d4d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42707 } } peers { permanent_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34219 } } peers { permanent_uuid: "42d8271920324faeb41b86b25273e068" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42877 } }
I20260812 07:08:17.717887 5169 raft_consensus.cc:374] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:17.717949 5169 raft_consensus.cc:729] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dfe309fa4f74e4f8bf3b74254e13b93, State: Initialized, Role: FOLLOWER
I20260812 07:08:17.717949 5166 raft_consensus.cc:348] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95f524d0c64e909978554f15374d4d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42707 } } peers { permanent_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34219 } } peers { permanent_uuid: "42d8271920324faeb41b86b25273e068" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42877 } }
I20260812 07:08:17.718238 5166 raft_consensus.cc:374] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:17.718315 5166 raft_consensus.cc:729] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95f524d0c64e909978554f15374d4d, State: Initialized, Role: FOLLOWER
I20260812 07:08:17.718463 5169 consensus_queue.cc:260] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95f524d0c64e909978554f15374d4d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42707 } } peers { permanent_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34219 } } peers { permanent_uuid: "42d8271920324faeb41b86b25273e068" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42877 } }
I20260812 07:08:17.718813 5166 consensus_queue.cc:260] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95f524d0c64e909978554f15374d4d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42707 } } peers { permanent_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34219 } } peers { permanent_uuid: "42d8271920324faeb41b86b25273e068" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42877 } }
I20260812 07:08:17.719352 4978 heartbeater.cc:503] Master 127.31.156.126:37441 was elected leader, sending a full tablet report...
I20260812 07:08:17.719826 5166 ts_tablet_manager.cc:1428] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 07:08:17.720214 4816 heartbeater.cc:503] Master 127.31.156.126:37441 was elected leader, sending a full tablet report...
I20260812 07:08:17.720721 5169 ts_tablet_manager.cc:1428] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260812 07:08:17.867959 4817 tablet.cc:2369] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:08:17.880558 5140 tablet.cc:2369] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:08:17.881062 4979 tablet.cc:2369] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:08:17.943878 5174 raft_consensus.cc:482] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:17.944118 5174 raft_consensus.cc:504] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95f524d0c64e909978554f15374d4d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42707 } } peers { permanent_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34219 } } peers { permanent_uuid: "42d8271920324faeb41b86b25273e068" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42877 } }
I20260812 07:08:17.945423 5174 leader_election.cc:288] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2dfe309fa4f74e4f8bf3b74254e13b93 (127.31.156.66:34219), 42d8271920324faeb41b86b25273e068 (127.31.156.67:42877)
I20260812 07:08:17.949657 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.945785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:59117 (local address 127.31.156.66:34219)
0812 07:08:17.945949 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.945953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.946258 (+ 305us) server_negotiation.cc:407] Connection header received
0812 07:08:17.946437 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.946441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.946492 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.946577 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:17.946914 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.947426 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.948646 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.949056 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.949192 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.949249 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.949440 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.949514 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:08:17.949694 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.945888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:57241 (local address 127.31.156.67:42877)
0812 07:08:17.946038 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.946044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.946330 (+ 286us) server_negotiation.cc:407] Connection header received
0812 07:08:17.946498 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.946500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.946549 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.946675 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:17.946987 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.947504 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.948646 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.949088 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.949247 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.949304 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.949493 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.949566 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:08:17.949708 5177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.945730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:34219 (local address 127.31.156.65:59117)
0812 07:08:17.946145 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:17.946158 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:17.946318 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:17.946592 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:17.946595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:17.946611 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:17.946791 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.946797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.947544 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.947548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.948510 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.948520 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.949193 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.949197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.949297 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.949328 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:08:17.949421 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 07:08:17.949498 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":49,"thread_start_us":157,"threads_started":1}
I20260812 07:08:17.949716 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.945769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:42877 (local address 127.31.156.65:57241)
0812 07:08:17.946214 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:17.946228 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:17.946367 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:17.946724 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:17.946727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:17.946744 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:17.946869 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.946874 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.947630 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.947634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.948512 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:17.948522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.949241 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.949244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:17.949365 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.949385 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 07:08:17.949474 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 07:08:17.949546 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":64,"thread_start_us":116,"threads_started":1}
I20260812 07:08:17.950484 5082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e2ec2e8386d47a198d4156a5e82475b" candidate_uuid: "5e95f524d0c64e909978554f15374d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42d8271920324faeb41b86b25273e068" is_pre_election: true
I20260812 07:08:17.950484 4919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e2ec2e8386d47a198d4156a5e82475b" candidate_uuid: "5e95f524d0c64e909978554f15374d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" is_pre_election: true
I20260812 07:08:17.950856 5082 raft_consensus.cc:2454] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95f524d0c64e909978554f15374d4d in term 0.
I20260812 07:08:17.950856 4919 raft_consensus.cc:2454] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95f524d0c64e909978554f15374d4d in term 0.
I20260812 07:08:17.951436 4680 leader_election.cc:302] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2dfe309fa4f74e4f8bf3b74254e13b93, 42d8271920324faeb41b86b25273e068, 5e95f524d0c64e909978554f15374d4d; no voters:
I20260812 07:08:17.951756 5174 raft_consensus.cc:2781] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:17.951890 5174 raft_consensus.cc:482] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:17.951956 5174 raft_consensus.cc:3037] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:17.953120 5174 raft_consensus.cc:504] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95f524d0c64e909978554f15374d4d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42707 } } peers { permanent_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34219 } } peers { permanent_uuid: "42d8271920324faeb41b86b25273e068" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42877 } }
I20260812 07:08:17.953510 5174 leader_election.cc:288] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [CANDIDATE]: Term 1 election: Requested vote from peers 2dfe309fa4f74e4f8bf3b74254e13b93 (127.31.156.66:34219), 42d8271920324faeb41b86b25273e068 (127.31.156.67:42877)
I20260812 07:08:17.954120 5082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e2ec2e8386d47a198d4156a5e82475b" candidate_uuid: "5e95f524d0c64e909978554f15374d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42d8271920324faeb41b86b25273e068"
I20260812 07:08:17.954284 5082 raft_consensus.cc:3037] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:17.954859 4919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e2ec2e8386d47a198d4156a5e82475b" candidate_uuid: "5e95f524d0c64e909978554f15374d4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93"
I20260812 07:08:17.954984 4919 raft_consensus.cc:3037] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:17.955642 5082 raft_consensus.cc:2454] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95f524d0c64e909978554f15374d4d in term 1.
I20260812 07:08:17.956029 4678 leader_election.cc:302] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42d8271920324faeb41b86b25273e068, 5e95f524d0c64e909978554f15374d4d; no voters:
I20260812 07:08:17.956229 5174 raft_consensus.cc:2781] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:17.956329 4919 raft_consensus.cc:2454] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95f524d0c64e909978554f15374d4d in term 1.
I20260812 07:08:17.956504 5174 raft_consensus.cc:686] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 1 LEADER]: Becoming Leader. State: Replica: 5e95f524d0c64e909978554f15374d4d, State: Running, Role: LEADER
I20260812 07:08:17.956966 5174 consensus_queue.cc:237] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95f524d0c64e909978554f15374d4d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42707 } } peers { permanent_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34219 } } peers { permanent_uuid: "42d8271920324faeb41b86b25273e068" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42877 } }
I20260812 07:08:17.960258 4588 catalog_manager.cc:5495] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e95f524d0c64e909978554f15374d4d (127.31.156.65). New cstate: current_term: 1 leader_uuid: "5e95f524d0c64e909978554f15374d4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95f524d0c64e909978554f15374d4d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42d8271920324faeb41b86b25273e068" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42877 } health_report { overall_health: UNKNOWN } } }
I20260812 07:08:17.987309 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.983480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43340 (local address 127.31.156.65:42707)
0812 07:08:17.983625 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.983629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.983642 (+ 13us) server_negotiation.cc:407] Connection header received
0812 07:08:17.983726 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.983729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.983789 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.983891 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:17.984145 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.984709 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.985638 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.985869 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.985963 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.986247 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:17.986858 (+ 611us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:17.986959 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.987110 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.987175 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:08:17.993813 4919 raft_consensus.cc:1261] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93 [term 1 FOLLOWER]: Refusing update from remote peer 5e95f524d0c64e909978554f15374d4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:17.993810 5082 raft_consensus.cc:1261] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068 [term 1 FOLLOWER]: Refusing update from remote peer 5e95f524d0c64e909978554f15374d4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:17.994576 5178 consensus_queue.cc:1035] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dfe309fa4f74e4f8bf3b74254e13b93" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:17.994797 5174 consensus_queue.cc:1035] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "42d8271920324faeb41b86b25273e068" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:18.006624 5183 mvcc.cc:205] Tried to move back new op lower bound from 7317579767775010816 to 7317579767634874368. Current Snapshot: MvccSnapshot[applied={T|T < 7317579767775010816 or (T in {7317579767775010816})}]
I20260812 07:08:18.013226 4588 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:18.013473 4588 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:18.016382 4588 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41427893a36642c1bafe7b60887cec19: Sending DeleteTablet for 3 replicas of tablet 4e2ec2e8386d47a198d4156a5e82475b
I20260812 07:08:18.017416 4733 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e2ec2e8386d47a198d4156a5e82475b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:18 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:43328
I20260812 07:08:18.017506 5057 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e2ec2e8386d47a198d4156a5e82475b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:18 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:43074
I20260812 07:08:18.018064 5196 tablet_replica.cc:330] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d: stopping tablet replica
I20260812 07:08:18.018045 4893 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e2ec2e8386d47a198d4156a5e82475b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:18 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:50560
I20260812 07:08:18.018483 5196 raft_consensus.cc:2229] T 4e2ec2e8386d47a198d4156a5e82475b P 5e95f524d0c64e909978554f15374d4d [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:18.018625 5198 tablet_replica.cc:330] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: stopping tablet replica
I20260812 07:08:18.019066 5197 tablet_replica.cc:330] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: stopping tablet replica
I20260812 07:08:18.019053 5198 raft_consensus.cc:2229] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:18.019376 5197 raft_consensus.cc:2229] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:18.019500 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 4662
I20260812 07:08:18.019654 5197 raft_consensus.cc:2258] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:18.023257 5197 ts_tablet_manager.cc:1905] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:18.027686 5197 ts_tablet_manager.cc:1918] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:18.027855 5197 log.cc:1196] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/4e2ec2e8386d47a198d4156a5e82475b
I20260812 07:08:18.028322 5197 ts_tablet_manager.cc:1939] T 4e2ec2e8386d47a198d4156a5e82475b P 42d8271920324faeb41b86b25273e068: Deleting consensus metadata
I20260812 07:08:18.030115 5198 raft_consensus.cc:2258] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:18.030325 4569 catalog_manager.cc:4841] TS 42d8271920324faeb41b86b25273e068 (127.31.156.67:42877): tablet 4e2ec2e8386d47a198d4156a5e82475b (table test-table [id=2527d42f55a74671bc38f1ee8f41410f]) successfully deleted
I20260812 07:08:18.032056 5198 ts_tablet_manager.cc:1905] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:18.035878 5198 ts_tablet_manager.cc:1918] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:18.036012 5198 log.cc:1196] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/4e2ec2e8386d47a198d4156a5e82475b
I20260812 07:08:18.036351 5198 ts_tablet_manager.cc:1939] T 4e2ec2e8386d47a198d4156a5e82475b P 2dfe309fa4f74e4f8bf3b74254e13b93: Deleting consensus metadata
I20260812 07:08:18.037959 4571 catalog_manager.cc:4841] TS 2dfe309fa4f74e4f8bf3b74254e13b93 (127.31.156.66:34219): tablet 4e2ec2e8386d47a198d4156a5e82475b (table test-table [id=2527d42f55a74671bc38f1ee8f41410f]) successfully deleted
W20260812 07:08:18.038334 4571 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:42707 (error 108)
I20260812 07:08:18.038647 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 4824
I20260812 07:08:18.042780 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:18.042234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:42707 (local address 127.0.0.1:43346)
0812 07:08:18.042433 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:18.042533 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260812 07:08:18.043000 4571 catalog_manager.cc:4584] TS 5e95f524d0c64e909978554f15374d4d (127.31.156.65:42707): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4e2ec2e8386d47a198d4156a5e82475b: Network error: Client connection negotiation failed: client connection to 127.31.156.65:42707: connect: Connection refused (error 111)
I20260812 07:08:18.048059 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 4983
I20260812 07:08:18.059417 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 4552
2026-08-12T07:08:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3272 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T07:08:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:18Z Disabled control of system clock
I20260812 07:08:18.096508 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:44177
--webserver_interface=127.31.156.126
--webserver_port=0
--builtin_ntp_servers=127.31.156.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:44177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 07:08:18.223378 5204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:18.223711 5204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:18.223752 5204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:18.228173 5204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:18.228263 5204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:18.228279 5204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:18.228297 5204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:18.228312 5204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:18.234122 5204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:44177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:44177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:18.235404 5204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:18.236877 5204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:18.246435 5213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:18.246435 5211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:18.246495 5215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:18.246516 5204 server_base.cc:1034] running on GCE node
I20260812 07:08:18.247632 5204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:18.248833 5204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:18.250065 5204 hybrid_clock.cc:648] HybridClock initialized: now 1786518498250009 us; error 72 us; skew 500 ppm
I20260812 07:08:18.256765 5204 webserver.cc:460] Webserver started at http://127.31.156.126:34143/ using document root <none> and password file <none>
I20260812 07:08:18.257498 5204 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:18.257618 5204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:18.257882 5204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:18.260264 5204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "bd2c57554cbe46b7af493ea54a43ca26"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "7ca0e5dc07d8b02ce205f4a9c89dcd5d"
server_key_iv: "9eeb7db12de4cb69081018cdc4eb07fd"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.260929 5204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "bd2c57554cbe46b7af493ea54a43ca26"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "7ca0e5dc07d8b02ce205f4a9c89dcd5d"
server_key_iv: "9eeb7db12de4cb69081018cdc4eb07fd"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.265241 5204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:18.268160 5222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:18.269886 5204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:08:18.270072 5204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "bd2c57554cbe46b7af493ea54a43ca26"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "7ca0e5dc07d8b02ce205f4a9c89dcd5d"
server_key_iv: "9eeb7db12de4cb69081018cdc4eb07fd"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.270231 5204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:18.275588 5204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:18.276293 5204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:18.276496 5204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:18.284870 5204 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:44177
I20260812 07:08:18.284909 5285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:44177 every 8 connection(s)
I20260812 07:08:18.286018 5204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:18.289995 5286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:18.294322 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 5204
I20260812 07:08:18.294481 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:18.294854 32369 external_mini_cluster.cc:1428] Setting key 568acff62df29a06c82fde83e2b7e777
I20260812 07:08:18.297366 5286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26: Bootstrap starting.
I20260812 07:08:18.300580 5286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:18.301597 5286 log.cc:824] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:18.301852 5289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:18.296576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.31.156.126:44177)
0812 07:08:18.297325 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:18.297343 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:18.297390 (+ 47us) server_negotiation.cc:407] Connection header received
0812 07:08:18.298180 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:18.298200 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:18.298796 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:18.299126 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:18.300174 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:18.300179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:18.300192 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:18.300204 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:18.300265 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:18.300290 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:18.300383 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:18.300600 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 07:08:18.300916 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":189,"threads_started":1}
I20260812 07:08:18.304070 5286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26: No bootstrap required, opened a new log
I20260812 07:08:18.308303 5286 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c57554cbe46b7af493ea54a43ca26" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44177 } }
I20260812 07:08:18.308614 5286 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:18.308703 5286 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd2c57554cbe46b7af493ea54a43ca26, State: Initialized, Role: FOLLOWER
I20260812 07:08:18.309188 5286 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c57554cbe46b7af493ea54a43ca26" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44177 } }
I20260812 07:08:18.309367 5286 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:18.309464 5286 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:18.309693 5286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:18.311033 5286 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c57554cbe46b7af493ea54a43ca26" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44177 } }
I20260812 07:08:18.311518 5286 leader_election.cc:302] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd2c57554cbe46b7af493ea54a43ca26; no voters:
I20260812 07:08:18.311960 5286 leader_election.cc:288] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:18.312088 5296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:18.312453 5296 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [term 1 LEADER]: Becoming Leader. State: Replica: bd2c57554cbe46b7af493ea54a43ca26, State: Running, Role: LEADER
I20260812 07:08:18.312801 5296 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c57554cbe46b7af493ea54a43ca26" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44177 } }
I20260812 07:08:18.313443 5286 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:18.314682 5299 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd2c57554cbe46b7af493ea54a43ca26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c57554cbe46b7af493ea54a43ca26" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44177 } } }
I20260812 07:08:18.314718 5298 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd2c57554cbe46b7af493ea54a43ca26. Latest consensus state: current_term: 1 leader_uuid: "bd2c57554cbe46b7af493ea54a43ca26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd2c57554cbe46b7af493ea54a43ca26" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 44177 } } }
I20260812 07:08:18.314821 5299 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:18.314821 5298 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:18.315258 5307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:18.318374 5307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:18.325006 5307 catalog_manager.cc:1347] Generated new cluster ID: f7bc958673df40c8a864adbf78953279
I20260812 07:08:18.325124 5307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:18.336918 5307 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:18.337344 5307 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:18.344334 5307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26: Generated new TSK 0
I20260812 07:08:18.345289 5307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:18.354219 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:44177
--builtin_ntp_servers=127.31.156.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 07:08:18.502611 5317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:18.503038 5317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:18.503122 5317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:18.507608 5317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:18.507697 5317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:18.507759 5317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:18.513670 5317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:44177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:18.515220 5317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:18.516813 5317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:08:18.528051 5317 server_base.cc:1034] running on GCE node
W20260812 07:08:18.528002 5326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:18.528008 5328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:18.528434 5323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:18.528957 5317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:18.529676 5317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:18.530876 5317 hybrid_clock.cc:648] HybridClock initialized: now 1786518498530859 us; error 34 us; skew 500 ppm
I20260812 07:08:18.533056 5317 webserver.cc:460] Webserver started at http://127.31.156.65:44555/ using document root <none> and password file <none>
I20260812 07:08:18.533746 5317 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:18.533849 5317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:18.534178 5317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:18.536481 5317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "cef477a47e624621bb3f5185de1e0983"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "05ec6453b0ad4cf6e5c800416f7d937f"
server_key_iv: "4d830e6ae661916a1ef38f3190dfeb42"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.537053 5317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "cef477a47e624621bb3f5185de1e0983"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "05ec6453b0ad4cf6e5c800416f7d937f"
server_key_iv: "4d830e6ae661916a1ef38f3190dfeb42"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.541465 5317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:18.544664 5335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:18.546296 5317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:18.546473 5317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "cef477a47e624621bb3f5185de1e0983"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "05ec6453b0ad4cf6e5c800416f7d937f"
server_key_iv: "4d830e6ae661916a1ef38f3190dfeb42"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.546631 5317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:18.552223 5317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:18.552989 5317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:18.553218 5317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:18.553814 5317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:18.555022 5317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:18.555114 5317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:18.555193 5317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:18.555229 5317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:18.566094 5317 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:34093
I20260812 07:08:18.566255 5473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:34093 every 8 connection(s)
I20260812 07:08:18.567399 5317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:18.573729 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 5317
I20260812 07:08:18.573909 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:08:18.574270 32369 external_mini_cluster.cc:1428] Setting key 2fc64e799a8766dccfe22a6b4557b955
I20260812 07:08:18.575654 5289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:18.569900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:48569 (local address 127.31.156.126:44177)
0812 07:08:18.570086 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:18.570092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:18.571480 (+ 1388us) server_negotiation.cc:407] Connection header received
0812 07:08:18.573109 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:18.573112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:18.573175 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:18.573268 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:18.575081 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:18.575087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:18.575095 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:18.575111 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:18.575155 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:18.575169 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:18.575273 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:18.575459 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 07:08:18.575513 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:08:18.576996 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:44177
--builtin_ntp_servers=127.31.156.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 07:08:18.577111 5477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:18.570436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:44177 (local address 127.31.156.65:48569)
0812 07:08:18.571278 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:18.571331 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:18.572797 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:18.573678 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:18.573688 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:18.573812 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:18.574697 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:18.574703 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:18.574779 (+ 76us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:18.574879 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:18.575258 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:18.575270 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:18.575278 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:08:18.575509 (+ 231us) client_negotiation.cc:240] Negotiation successful
0812 07:08:18.575832 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":196,"threads_started":1}
I20260812 07:08:18.578634 5475 heartbeater.cc:347] Connected to a master server at 127.31.156.126:44177
I20260812 07:08:18.578984 5475 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:18.579547 5475 heartbeater.cc:511] Master 127.31.156.126:44177 requested a full tablet report, sending...
I20260812 07:08:18.581411 5242 ts_manager.cc:194] Registered new tserver with Master: cef477a47e624621bb3f5185de1e0983 (127.31.156.65:34093)
W20260812 07:08:18.702826 5478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:18.703138 5478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:18.703181 5478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:18.707468 5478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:18.707549 5478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:18.707610 5478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:18.712738 5478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:44177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:18.714488 5478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:18.716733 5478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:18.727739 5484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:18.727746 5487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:18.727746 5483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:18.728322 5478 server_base.cc:1034] running on GCE node
I20260812 07:08:18.728719 5478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:18.729372 5478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:18.730578 5478 hybrid_clock.cc:648] HybridClock initialized: now 1786518498730529 us; error 62 us; skew 500 ppm
I20260812 07:08:18.733116 5478 webserver.cc:460] Webserver started at http://127.31.156.66:35213/ using document root <none> and password file <none>
I20260812 07:08:18.733793 5478 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:18.733865 5478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:18.734059 5478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:18.736392 5478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "958c895b0a3a4d109dda4db6aeaa7762"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "b156eb14da6697fa405c653c6cef0470"
server_key_iv: "082c95dadff5118594e257d1cfaff356"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.736965 5478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "958c895b0a3a4d109dda4db6aeaa7762"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "b156eb14da6697fa405c653c6cef0470"
server_key_iv: "082c95dadff5118594e257d1cfaff356"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.741333 5478 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:18.744385 5498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:18.745872 5478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:08:18.746032 5478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "958c895b0a3a4d109dda4db6aeaa7762"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "b156eb14da6697fa405c653c6cef0470"
server_key_iv: "082c95dadff5118594e257d1cfaff356"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.746172 5478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:18.751487 5478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:18.752250 5478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:18.752475 5478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:18.753110 5478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:18.754171 5478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:18.754249 5478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:18.754323 5478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:18.754370 5478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:18.764423 5478 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:36373
I20260812 07:08:18.764456 5635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:36373 every 8 connection(s)
I20260812 07:08:18.765463 5478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:18.767194 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 5478
I20260812 07:08:18.767344 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:08:18.767786 32369 external_mini_cluster.cc:1428] Setting key 9b7cc13ef04cbdd06a764f1646c52e5a
I20260812 07:08:18.770130 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:44177
--builtin_ntp_servers=127.31.156.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 07:08:18.772388 5289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:18.767583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:60715 (local address 127.31.156.126:44177)
0812 07:08:18.767750 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:18.767756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:18.768988 (+ 1232us) server_negotiation.cc:407] Connection header received
0812 07:08:18.770393 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:18.770397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:18.770444 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:18.770489 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:18.771886 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:18.771890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:18.771893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:18.771906 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:18.771933 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:18.771941 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:18.772056 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:18.772195 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 07:08:18.772244 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:08:18.773331 5638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:18.768084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:44177 (local address 127.31.156.66:60715)
0812 07:08:18.768817 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:18.768873 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:18.770130 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:18.770868 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:18.770875 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:18.770966 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:18.771619 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:18.771620 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:18.771663 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:18.771734 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:18.772027 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:18.772037 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:18.772044 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:08:18.772241 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 07:08:18.772511 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":135,"threads_started":1}
I20260812 07:08:18.774628 5636 heartbeater.cc:347] Connected to a master server at 127.31.156.126:44177
I20260812 07:08:18.774940 5636 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:18.775401 5636 heartbeater.cc:511] Master 127.31.156.126:44177 requested a full tablet report, sending...
I20260812 07:08:18.776556 5242 ts_manager.cc:194] Registered new tserver with Master: 958c895b0a3a4d109dda4db6aeaa7762 (127.31.156.66:36373)
W20260812 07:08:18.899641 5640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:18.899982 5640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:18.900027 5640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:18.904426 5640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:18.904511 5640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:18.904574 5640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:18.910002 5640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:44177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:18.911480 5640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:18.913455 5640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:18.927651 5648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:18.927814 5651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:18.927692 5647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:18.927954 5640 server_base.cc:1034] running on GCE node
I20260812 07:08:18.928893 5640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:18.929697 5640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:18.930963 5640 hybrid_clock.cc:648] HybridClock initialized: now 1786518498930934 us; error 36 us; skew 500 ppm
I20260812 07:08:18.933466 5640 webserver.cc:460] Webserver started at http://127.31.156.67:33213/ using document root <none> and password file <none>
I20260812 07:08:18.934207 5640 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:18.934321 5640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:18.934602 5640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:18.937783 5640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "7347cef80841456ca345aba55875e553"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "cd6271d27739c1d3c78c73ecc8f8aa8d"
server_key_iv: "2d9f6d36aadfe6d80b0eca248a5efd11"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.938508 5640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "7347cef80841456ca345aba55875e553"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "cd6271d27739c1d3c78c73ecc8f8aa8d"
server_key_iv: "2d9f6d36aadfe6d80b0eca248a5efd11"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.943463 5640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 07:08:18.946827 5658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:18.948666 5640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:08:18.948872 5640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "7347cef80841456ca345aba55875e553"
format_stamp: "Formatted at 2026-08-12 07:08:18 on dist-test-slave-csg5"
server_key: "cd6271d27739c1d3c78c73ecc8f8aa8d"
server_key_iv: "2d9f6d36aadfe6d80b0eca248a5efd11"
server_key_version: "encryptionkey@0"
I20260812 07:08:18.949039 5640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:18.955102 5640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:18.955930 5640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:18.956161 5640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:18.956821 5640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:18.957937 5640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:18.958027 5640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:18.958134 5640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:18.958189 5640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:18.967974 5640 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:39319
I20260812 07:08:18.968115 5796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:39319 every 8 connection(s)
I20260812 07:08:18.969048 5640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:18.976763 5289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:18.971407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:59319 (local address 127.31.156.126:44177)
0812 07:08:18.971572 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:18.971576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:18.973058 (+ 1482us) server_negotiation.cc:407] Connection header received
0812 07:08:18.974712 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:18.974716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:18.974776 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:18.974891 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:18.976171 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:18.976176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:18.976178 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:18.976195 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:18.976223 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:18.976236 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:18.976369 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:18.976517 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 07:08:18.976568 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:08:18.977702 5799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:18.971878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:44177 (local address 127.31.156.67:59319)
0812 07:08:18.972877 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:18.972923 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:18.974402 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:18.975085 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:18.975092 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:18.975186 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:18.975867 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:18.975869 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:18.975914 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:18.975986 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:18.976340 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:18.976350 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:18.976357 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:08:18.976557 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 07:08:18.976820 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":212,"threads_started":1}
I20260812 07:08:18.979220 5797 heartbeater.cc:347] Connected to a master server at 127.31.156.126:44177
I20260812 07:08:18.979449 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 5640
I20260812 07:08:18.979480 5797 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:18.979578 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:08:18.979878 32369 external_mini_cluster.cc:1428] Setting key e7485bf85d13ebf9eda659c6e2d280a7
I20260812 07:08:18.980000 5797 heartbeater.cc:511] Master 127.31.156.126:44177 requested a full tablet report, sending...
I20260812 07:08:18.981324 5242 ts_manager.cc:194] Registered new tserver with Master: 7347cef80841456ca345aba55875e553 (127.31.156.67:39319)
I20260812 07:08:18.982299 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:08:18.986888 5289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:18.984525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55004 (local address 127.31.156.126:44177)
0812 07:08:18.984702 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:18.984708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:18.984955 (+ 247us) server_negotiation.cc:407] Connection header received
0812 07:08:18.985007 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:18.985009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:18.985052 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:18.985154 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:18.986151 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:18.986154 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:18.986157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:18.986170 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:18.986191 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:18.986200 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:18.986376 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:18.986618 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 07:08:18.986690 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:08:18.990906 5240 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:08:18.993963 5240 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:19.007931 5811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.004241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:39319 (local address 127.0.0.1:43088)
0812 07:08:19.005214 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:19.005270 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:19.005380 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:19.006109 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:19.006112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:19.006128 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:19.006731 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:19.006733 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:19.006757 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:19.006852 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:19.007526 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:19.007537 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:19.007543 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:08:19.007664 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 07:08:19.007769 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":119,"threads_started":1}
I20260812 07:08:19.008018 5814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.004496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41372 (local address 127.31.156.66:36373)
0812 07:08:19.004981 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:19.004996 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:19.005485 (+ 489us) server_negotiation.cc:407] Connection header received
0812 07:08:19.005675 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:19.005682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:19.005871 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:19.005984 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:19.007368 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:19.007373 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:19.007382 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:19.007396 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:19.007433 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:19.007470 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:19.007677 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:19.007754 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 07:08:19.007844 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":105,"threads_started":1}
I20260812 07:08:19.008242 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.004214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38404 (local address 127.31.156.65:34093)
0812 07:08:19.005217 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:19.005230 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:19.005500 (+ 270us) server_negotiation.cc:407] Connection header received
0812 07:08:19.005566 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:19.005572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:19.005777 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:19.005942 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:19.007230 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:19.007241 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:19.007273 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:19.007291 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:19.007409 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:19.007449 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:19.007567 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:19.007999 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 07:08:19.008079 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903,"thread_start_us":137,"threads_started":1}
I20260812 07:08:19.008479 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.004055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34093 (local address 127.0.0.1:38404)
0812 07:08:19.005211 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:19.005269 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:19.005389 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:19.005948 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:19.005955 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:19.006032 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:19.006840 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:19.006842 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:19.006851 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:19.006875 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:19.007743 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:19.007747 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:19.007750 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 07:08:19.007844 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 07:08:19.007878 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"spinlock_wait_cycles":2816,"thread_start_us":74,"threads_started":1}
I20260812 07:08:19.008702 5815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.004055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:36373 (local address 127.0.0.1:41372)
0812 07:08:19.005383 (+ 1328us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:19.005396 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:19.005546 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:19.006773 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:19.006776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:19.006789 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:19.007160 (+ 371us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:19.007162 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:19.007172 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:19.007198 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:19.007619 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:19.007625 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:19.007629 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 07:08:19.007691 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 07:08:19.008488 (+ 797us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1258,"mutex_wait_us":1437,"thread_start_us":88,"threads_started":1}
I20260812 07:08:19.009104 5816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.004786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43088 (local address 127.31.156.67:39319)
0812 07:08:19.005678 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:19.005691 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:19.005713 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:08:19.005760 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:19.005764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:19.005958 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:19.006368 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:19.007363 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:19.007367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:19.007375 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:19.007384 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:19.007415 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:19.007431 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:19.007562 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:19.007803 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 07:08:19.007888 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":780,"thread_start_us":190,"threads_started":1}
I20260812 07:08:19.011633 5392 tablet_service.cc:1467] Processing CreateTablet for tablet ef5b89cb88e24fd2a8e25fc040d1fce5 (DEFAULT_TABLE table=test-table [id=3809c5e4c1344b679fb1f4e078375d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:19.011895 5558 tablet_service.cc:1467] Processing CreateTablet for tablet ef5b89cb88e24fd2a8e25fc040d1fce5 (DEFAULT_TABLE table=test-table [id=3809c5e4c1344b679fb1f4e078375d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:19.011895 5714 tablet_service.cc:1467] Processing CreateTablet for tablet ef5b89cb88e24fd2a8e25fc040d1fce5 (DEFAULT_TABLE table=test-table [id=3809c5e4c1344b679fb1f4e078375d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:19.012917 5392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef5b89cb88e24fd2a8e25fc040d1fce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:19.012991 5558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef5b89cb88e24fd2a8e25fc040d1fce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:19.012991 5714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef5b89cb88e24fd2a8e25fc040d1fce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:19.022460 5820 tablet_bootstrap.cc:492] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553: Bootstrap starting.
I20260812 07:08:19.022961 5817 tablet_bootstrap.cc:492] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983: Bootstrap starting.
I20260812 07:08:19.023329 5818 tablet_bootstrap.cc:492] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762: Bootstrap starting.
I20260812 07:08:19.025645 5818 tablet_bootstrap.cc:654] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:19.025722 5817 tablet_bootstrap.cc:654] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:19.025723 5820 tablet_bootstrap.cc:654] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:19.026530 5818 log.cc:824] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:19.026636 5817 log.cc:824] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:19.026636 5820 log.cc:824] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:19.029263 5817 tablet_bootstrap.cc:492] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983: No bootstrap required, opened a new log
I20260812 07:08:19.029263 5818 tablet_bootstrap.cc:492] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762: No bootstrap required, opened a new log
I20260812 07:08:19.029515 5817 ts_tablet_manager.cc:1397] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260812 07:08:19.029506 5818 ts_tablet_manager.cc:1397] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260812 07:08:19.030109 5820 tablet_bootstrap.cc:492] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553: No bootstrap required, opened a new log
I20260812 07:08:19.030383 5820 ts_tablet_manager.cc:1397] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260812 07:08:19.034302 5820 raft_consensus.cc:348] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cef477a47e624621bb3f5185de1e0983" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34093 } } peers { permanent_uuid: "958c895b0a3a4d109dda4db6aeaa7762" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36373 } } peers { permanent_uuid: "7347cef80841456ca345aba55875e553" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 39319 } }
I20260812 07:08:19.034632 5820 raft_consensus.cc:374] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:19.034726 5820 raft_consensus.cc:729] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7347cef80841456ca345aba55875e553, State: Initialized, Role: FOLLOWER
I20260812 07:08:19.034576 5817 raft_consensus.cc:348] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cef477a47e624621bb3f5185de1e0983" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34093 } } peers { permanent_uuid: "958c895b0a3a4d109dda4db6aeaa7762" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36373 } } peers { permanent_uuid: "7347cef80841456ca345aba55875e553" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 39319 } }
I20260812 07:08:19.034576 5818 raft_consensus.cc:348] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cef477a47e624621bb3f5185de1e0983" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34093 } } peers { permanent_uuid: "958c895b0a3a4d109dda4db6aeaa7762" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36373 } } peers { permanent_uuid: "7347cef80841456ca345aba55875e553" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 39319 } }
I20260812 07:08:19.034898 5817 raft_consensus.cc:374] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:19.034898 5818 raft_consensus.cc:374] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:19.034965 5817 raft_consensus.cc:729] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cef477a47e624621bb3f5185de1e0983, State: Initialized, Role: FOLLOWER
I20260812 07:08:19.034965 5818 raft_consensus.cc:729] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 958c895b0a3a4d109dda4db6aeaa7762, State: Initialized, Role: FOLLOWER
I20260812 07:08:19.035354 5820 consensus_queue.cc:260] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cef477a47e624621bb3f5185de1e0983" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34093 } } peers { permanent_uuid: "958c895b0a3a4d109dda4db6aeaa7762" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36373 } } peers { permanent_uuid: "7347cef80841456ca345aba55875e553" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 39319 } }
I20260812 07:08:19.035431 5817 consensus_queue.cc:260] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cef477a47e624621bb3f5185de1e0983" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34093 } } peers { permanent_uuid: "958c895b0a3a4d109dda4db6aeaa7762" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36373 } } peers { permanent_uuid: "7347cef80841456ca345aba55875e553" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 39319 } }
I20260812 07:08:19.035431 5818 consensus_queue.cc:260] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cef477a47e624621bb3f5185de1e0983" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34093 } } peers { permanent_uuid: "958c895b0a3a4d109dda4db6aeaa7762" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36373 } } peers { permanent_uuid: "7347cef80841456ca345aba55875e553" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 39319 } }
I20260812 07:08:19.036654 5817 ts_tablet_manager.cc:1428] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:08:19.036655 5820 ts_tablet_manager.cc:1428] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260812 07:08:19.036654 5818 ts_tablet_manager.cc:1428] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:08:19.037077 5475 heartbeater.cc:503] Master 127.31.156.126:44177 was elected leader, sending a full tablet report...
I20260812 07:08:19.037079 5797 heartbeater.cc:503] Master 127.31.156.126:44177 was elected leader, sending a full tablet report...
I20260812 07:08:19.037079 5636 heartbeater.cc:503] Master 127.31.156.126:44177 was elected leader, sending a full tablet report...
W20260812 07:08:19.068869 5476 tablet.cc:2369] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:08:19.139117 5826 raft_consensus.cc:482] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:19.139413 5826 raft_consensus.cc:504] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cef477a47e624621bb3f5185de1e0983" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34093 } } peers { permanent_uuid: "958c895b0a3a4d109dda4db6aeaa7762" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36373 } } peers { permanent_uuid: "7347cef80841456ca345aba55875e553" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 39319 } }
I20260812 07:08:19.140862 5826 leader_election.cc:288] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 958c895b0a3a4d109dda4db6aeaa7762 (127.31.156.66:36373), 7347cef80841456ca345aba55875e553 (127.31.156.67:39319)
I20260812 07:08:19.143872 5829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.141239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:39319 (local address 127.31.156.65:48225)
0812 07:08:19.141585 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:19.141600 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:19.141719 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:19.142119 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:19.142122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:19.142144 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:19.142981 (+ 837us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:19.142985 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:19.143000 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:19.143039 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:19.143408 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:19.143410 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:19.143413 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 07:08:19.143627 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 07:08:19.143678 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"spinlock_wait_cycles":12800,"thread_start_us":72,"threads_started":1}
I20260812 07:08:19.143889 5816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.141498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:48225 (local address 127.31.156.67:39319)
0812 07:08:19.141650 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:19.141654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:19.141670 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:08:19.141885 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:19.141889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:19.141937 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:19.142049 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:19.143222 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:19.143226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:19.143229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:19.143245 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:19.143272 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:19.143285 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:19.143403 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:19.143639 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 07:08:19.143750 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:08:19.143980 5814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.141256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:37963 (local address 127.31.156.66:36373)
0812 07:08:19.141402 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:19.141407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:19.141610 (+ 203us) server_negotiation.cc:407] Connection header received
0812 07:08:19.141887 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:19.141891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:19.141937 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:19.142049 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:19.143327 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:19.143333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:19.143337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:19.143355 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:19.143382 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:19.143392 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:19.143497 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:19.143790 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 07:08:19.143835 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:08:19.143905 5828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.141118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:36373 (local address 127.31.156.65:37963)
0812 07:08:19.141508 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:19.141525 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:19.141719 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:19.142119 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:19.142122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:19.142144 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:19.143007 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:19.143009 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:19.143017 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:19.143039 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:19.143586 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:19.143591 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:19.143597 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:08:19.143755 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 07:08:19.143785 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"spinlock_wait_cycles":29312,"thread_start_us":114,"threads_started":1}
I20260812 07:08:19.144615 5739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef5b89cb88e24fd2a8e25fc040d1fce5" candidate_uuid: "cef477a47e624621bb3f5185de1e0983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7347cef80841456ca345aba55875e553" is_pre_election: true
I20260812 07:08:19.144740 5581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef5b89cb88e24fd2a8e25fc040d1fce5" candidate_uuid: "cef477a47e624621bb3f5185de1e0983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "958c895b0a3a4d109dda4db6aeaa7762" is_pre_election: true
I20260812 07:08:19.145020 5739 raft_consensus.cc:2454] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cef477a47e624621bb3f5185de1e0983 in term 0.
I20260812 07:08:19.145094 5581 raft_consensus.cc:2454] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cef477a47e624621bb3f5185de1e0983 in term 0.
I20260812 07:08:19.145650 5338 leader_election.cc:302] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7347cef80841456ca345aba55875e553, cef477a47e624621bb3f5185de1e0983; no voters:
I20260812 07:08:19.146024 5826 raft_consensus.cc:2781] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:19.146137 5826 raft_consensus.cc:482] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:19.146185 5826 raft_consensus.cc:3037] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:19.147440 5826 raft_consensus.cc:504] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cef477a47e624621bb3f5185de1e0983" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34093 } } peers { permanent_uuid: "958c895b0a3a4d109dda4db6aeaa7762" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36373 } } peers { permanent_uuid: "7347cef80841456ca345aba55875e553" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 39319 } }
I20260812 07:08:19.147895 5826 leader_election.cc:288] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [CANDIDATE]: Term 1 election: Requested vote from peers 958c895b0a3a4d109dda4db6aeaa7762 (127.31.156.66:36373), 7347cef80841456ca345aba55875e553 (127.31.156.67:39319)
I20260812 07:08:19.148352 5581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef5b89cb88e24fd2a8e25fc040d1fce5" candidate_uuid: "cef477a47e624621bb3f5185de1e0983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "958c895b0a3a4d109dda4db6aeaa7762"
I20260812 07:08:19.148499 5739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef5b89cb88e24fd2a8e25fc040d1fce5" candidate_uuid: "cef477a47e624621bb3f5185de1e0983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7347cef80841456ca345aba55875e553"
I20260812 07:08:19.148527 5581 raft_consensus.cc:3037] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:19.148661 5739 raft_consensus.cc:3037] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:19.149812 5581 raft_consensus.cc:2454] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cef477a47e624621bb3f5185de1e0983 in term 1.
I20260812 07:08:19.149938 5739 raft_consensus.cc:2454] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cef477a47e624621bb3f5185de1e0983 in term 1.
I20260812 07:08:19.150326 5339 leader_election.cc:302] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 958c895b0a3a4d109dda4db6aeaa7762, cef477a47e624621bb3f5185de1e0983; no voters:
I20260812 07:08:19.150569 5826 raft_consensus.cc:2781] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:19.150882 5826 raft_consensus.cc:686] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 1 LEADER]: Becoming Leader. State: Replica: cef477a47e624621bb3f5185de1e0983, State: Running, Role: LEADER
I20260812 07:08:19.151312 5826 consensus_queue.cc:237] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cef477a47e624621bb3f5185de1e0983" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34093 } } peers { permanent_uuid: "958c895b0a3a4d109dda4db6aeaa7762" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36373 } } peers { permanent_uuid: "7347cef80841456ca345aba55875e553" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 39319 } }
I20260812 07:08:19.155280 5239 catalog_manager.cc:5495] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 reported cstate change: term changed from 0 to 1, leader changed from <none> to cef477a47e624621bb3f5185de1e0983 (127.31.156.65). New cstate: current_term: 1 leader_uuid: "cef477a47e624621bb3f5185de1e0983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cef477a47e624621bb3f5185de1e0983" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "958c895b0a3a4d109dda4db6aeaa7762" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7347cef80841456ca345aba55875e553" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 39319 } health_report { overall_health: UNKNOWN } } }
I20260812 07:08:19.171196 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.169317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38414 (local address 127.31.156.65:34093)
0812 07:08:19.169539 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:19.169543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:19.169559 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:08:19.169660 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:19.169662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:19.169732 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:19.169812 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:19.170650 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:19.170653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:19.170655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:19.170700 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:19.170723 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:19.170732 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:19.170827 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:19.170933 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 07:08:19.170975 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260812 07:08:19.178377 5739 raft_consensus.cc:1261] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553 [term 1 FOLLOWER]: Refusing update from remote peer cef477a47e624621bb3f5185de1e0983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:19.178421 5581 raft_consensus.cc:1261] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762 [term 1 FOLLOWER]: Refusing update from remote peer cef477a47e624621bb3f5185de1e0983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:19.179308 5826 consensus_queue.cc:1035] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "958c895b0a3a4d109dda4db6aeaa7762" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:19.179603 5831 consensus_queue.cc:1035] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7347cef80841456ca345aba55875e553" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 39319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:19.192358 5837 mvcc.cc:205] Tried to move back new op lower bound from 7317579772625059840 to 7317579772527337472. Current Snapshot: MvccSnapshot[applied={T|T < 7317579772625059840}]
I20260812 07:08:19.197664 5239 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:19.197913 5239 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:19.201130 5239 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd2c57554cbe46b7af493ea54a43ca26: Sending DeleteTablet for 3 replicas of tablet ef5b89cb88e24fd2a8e25fc040d1fce5
I20260812 07:08:19.202201 5558 tablet_service.cc:1514] Processing DeleteTablet for tablet ef5b89cb88e24fd2a8e25fc040d1fce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:19 UTC) from {username='slave'} at 127.0.0.1:41372
I20260812 07:08:19.202188 5392 tablet_service.cc:1514] Processing DeleteTablet for tablet ef5b89cb88e24fd2a8e25fc040d1fce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:19 UTC) from {username='slave'} at 127.0.0.1:38404
I20260812 07:08:19.202939 5849 tablet_replica.cc:330] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983: stopping tablet replica
I20260812 07:08:19.202955 5714 tablet_service.cc:1514] Processing DeleteTablet for tablet ef5b89cb88e24fd2a8e25fc040d1fce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:19 UTC) from {username='slave'} at 127.0.0.1:43088
I20260812 07:08:19.203356 5849 raft_consensus.cc:2229] T ef5b89cb88e24fd2a8e25fc040d1fce5 P cef477a47e624621bb3f5185de1e0983 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:19.203424 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 5317
I20260812 07:08:19.204185 5850 tablet_replica.cc:330] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762: stopping tablet replica
I20260812 07:08:19.204464 5850 raft_consensus.cc:2229] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:19.204767 5850 raft_consensus.cc:2258] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:19.206744 5851 tablet_replica.cc:330] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553: stopping tablet replica
I20260812 07:08:19.206756 5850 ts_tablet_manager.cc:1905] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:19.207136 5851 raft_consensus.cc:2229] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:19.207617 5851 raft_consensus.cc:2258] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:19.209810 5851 ts_tablet_manager.cc:1905] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:19.210071 5850 ts_tablet_manager.cc:1918] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:19.210188 5850 log.cc:1196] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/ef5b89cb88e24fd2a8e25fc040d1fce5
I20260812 07:08:19.210568 5850 ts_tablet_manager.cc:1939] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 958c895b0a3a4d109dda4db6aeaa7762: Deleting consensus metadata
I20260812 07:08:19.212250 5226 catalog_manager.cc:4841] TS 958c895b0a3a4d109dda4db6aeaa7762 (127.31.156.66:36373): tablet ef5b89cb88e24fd2a8e25fc040d1fce5 (table test-table [id=3809c5e4c1344b679fb1f4e078375d90]) successfully deleted
I20260812 07:08:19.213013 5851 ts_tablet_manager.cc:1918] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:19.213109 5851 log.cc:1196] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/ef5b89cb88e24fd2a8e25fc040d1fce5
I20260812 07:08:19.213441 5851 ts_tablet_manager.cc:1939] T ef5b89cb88e24fd2a8e25fc040d1fce5 P 7347cef80841456ca345aba55875e553: Deleting consensus metadata
I20260812 07:08:19.215113 5225 catalog_manager.cc:4841] TS 7347cef80841456ca345aba55875e553 (127.31.156.67:39319): tablet ef5b89cb88e24fd2a8e25fc040d1fce5 (table test-table [id=3809c5e4c1344b679fb1f4e078375d90]) successfully deleted
W20260812 07:08:19.216521 5223 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:34093 (error 108)
I20260812 07:08:19.216730 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 5478
I20260812 07:08:19.217988 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.217569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34093 (local address 127.0.0.1:38426)
0812 07:08:19.217806 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:19.217881 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:34093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260812 07:08:19.218163 5223 catalog_manager.cc:4584] TS cef477a47e624621bb3f5185de1e0983 (127.31.156.65:34093): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef5b89cb88e24fd2a8e25fc040d1fce5: Network error: Client connection negotiation failed: client connection to 127.31.156.65:34093: connect: Connection refused (error 111)
I20260812 07:08:19.227701 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 5640
I20260812 07:08:19.237285 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 5204
2026-08-12T07:08:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1177 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 07:08:19.264606 32369 mini_oidc.cc:152] Starting JWKS server
I20260812 07:08:19.267035 32369 webserver.cc:460] Webserver started at https://127.0.0.1:46789/ using document root <none> and password file <none>
I20260812 07:08:19.267236 32369 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 07:08:19.267525 32369 webserver.cc:460] Webserver started at http://127.0.0.1:33077/ using document root <none> and password file <none>
2026-08-12T07:08:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:19Z Disabled control of system clock
I20260812 07:08:19.281142 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:41585
--webserver_interface=127.31.156.126
--webserver_port=0
--builtin_ntp_servers=127.31.156.84:37647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:41585
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46789/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/testchainca.cert with env {}
W20260812 07:08:19.417865 5859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:19.418200 5859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:19.418242 5859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:19.422530 5859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:19.422616 5859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:19.422633 5859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:19.422673 5859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:19.422690 5859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:08:19.422708 5859 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:08:19.422720 5859 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46789/jwks.json
I20260812 07:08:19.428522 5859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:37647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:41585
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:41585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46789/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:19.429806 5859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:19.430894 5859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:19.442070 5869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:19.442226 5867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:19.442080 5866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:19.442585 5859 server_base.cc:1034] running on GCE node
I20260812 07:08:19.443357 5859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:19.444660 5859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:19.445863 5859 hybrid_clock.cc:648] HybridClock initialized: now 1786518499445847 us; error 37 us; skew 500 ppm
I20260812 07:08:19.448513 5859 webserver.cc:460] Webserver started at http://127.31.156.126:40959/ using document root <none> and password file <none>
I20260812 07:08:19.449209 5859 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:19.449429 5859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:19.449816 5859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:19.452886 5859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "2f5dd27b271740b4bc6827a7473e471f"
format_stamp: "Formatted at 2026-08-12 07:08:19 on dist-test-slave-csg5"
server_key: "a897f678a8ca7880b2541ab32f7efeef"
server_key_iv: "51d3b801b6993c2f09bc36896b2eadab"
server_key_version: "encryptionkey@0"
I20260812 07:08:19.453672 5859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "2f5dd27b271740b4bc6827a7473e471f"
format_stamp: "Formatted at 2026-08-12 07:08:19 on dist-test-slave-csg5"
server_key: "a897f678a8ca7880b2541ab32f7efeef"
server_key_iv: "51d3b801b6993c2f09bc36896b2eadab"
server_key_version: "encryptionkey@0"
I20260812 07:08:19.459045 5859 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260812 07:08:19.462493 5877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:19.464326 5859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:19.464600 5859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "2f5dd27b271740b4bc6827a7473e471f"
format_stamp: "Formatted at 2026-08-12 07:08:19 on dist-test-slave-csg5"
server_key: "a897f678a8ca7880b2541ab32f7efeef"
server_key_iv: "51d3b801b6993c2f09bc36896b2eadab"
server_key_version: "encryptionkey@0"
I20260812 07:08:19.464807 5859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:19.492044 5859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:19.492897 5859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:19.493149 5859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:19.501233 5859 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:41585
I20260812 07:08:19.501245 5940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:41585 every 8 connection(s)
I20260812 07:08:19.502357 5859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:19.505929 5941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:19.510926 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 5859
I20260812 07:08:19.511094 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:19.511413 32369 external_mini_cluster.cc:1428] Setting key 82bddc5282e052aa987e30990554d4c5
I20260812 07:08:19.513352 5941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f: Bootstrap starting.
I20260812 07:08:19.516283 5941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:19.517185 5941 log.cc:824] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:19.520438 5941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f: No bootstrap required, opened a new log
I20260812 07:08:19.522392 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.513166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.31.156.126:41585)
0812 07:08:19.513868 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:19.513887 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:19.513934 (+ 47us) server_negotiation.cc:407] Connection header received
0812 07:08:19.514766 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:19.514787 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:19.515234 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:19.515540 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:19.516026 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.517288 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.518569 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.519180 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.519338 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:19.520318 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:19.520323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:19.520338 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:19.520836 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:19.520902 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:19.520925 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:19.521018 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:19.521206 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 07:08:19.521606 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":160,"threads_started":1}
I20260812 07:08:19.525436 5941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5dd27b271740b4bc6827a7473e471f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41585 } }
I20260812 07:08:19.525812 5941 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:19.525908 5941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f5dd27b271740b4bc6827a7473e471f, State: Initialized, Role: FOLLOWER
I20260812 07:08:19.526012 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:08:19.526579 5941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5dd27b271740b4bc6827a7473e471f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41585 } }
I20260812 07:08:19.526814 5941 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:19.526870 5941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:19.526950 5941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:19.528201 5941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5dd27b271740b4bc6827a7473e471f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41585 } }
I20260812 07:08:19.528620 5941 leader_election.cc:302] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f5dd27b271740b4bc6827a7473e471f; no voters:
I20260812 07:08:19.528986 5941 leader_election.cc:288] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:19.529202 5947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:19.529469 5947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [term 1 LEADER]: Becoming Leader. State: Replica: 2f5dd27b271740b4bc6827a7473e471f, State: Running, Role: LEADER
I20260812 07:08:19.529842 5947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5dd27b271740b4bc6827a7473e471f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41585 } }
I20260812 07:08:19.530581 5941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:19.531684 5949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f5dd27b271740b4bc6827a7473e471f. Latest consensus state: current_term: 1 leader_uuid: "2f5dd27b271740b4bc6827a7473e471f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5dd27b271740b4bc6827a7473e471f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41585 } } }
I20260812 07:08:19.531725 5948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f5dd27b271740b4bc6827a7473e471f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5dd27b271740b4bc6827a7473e471f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41585 } } }
I20260812 07:08:19.531872 5949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:19.531872 5948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:19.532342 5959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:19.535811 5959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:19.542306 5959 catalog_manager.cc:1347] Generated new cluster ID: 389558ddfc384ec38374669389bc47ec
I20260812 07:08:19.542402 5959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:19.560099 5959 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:19.561183 5959 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:19.569438 5959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f5dd27b271740b4bc6827a7473e471f: Generated new TSK 0
I20260812 07:08:19.570353 5959 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 07:08:19.617156 5977 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 07:08:19.617350 5977 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260812 07:08:19.619113 5976 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:19.589405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41585 (local address 127.0.0.1:44752)
0812 07:08:19.589869 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:19.589888 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:19.590054 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:19.590457 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:19.590461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:19.590471 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:08:19.590625 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:19.590634 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.591605 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.591609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:19.592762 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:19.592775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.593318 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.593322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:19.593434 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:19.593505 (+ 71us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:08:19.617611 (+ 24106us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 07:08:19.617837 (+ 226us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.156.126:41585: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":136,"threads_started":1}
I20260812 07:08:19.619812 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.589608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.31.156.126:41585)
0812 07:08:19.589776 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:19.589780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:19.589980 (+ 200us) server_negotiation.cc:407] Connection header received
0812 07:08:19.590253 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:19.590255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:19.590312 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:19.590410 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:08:19.590836 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.591456 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.592914 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.593176 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.593292 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:19.594004 (+ 712us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:08:19.617252 (+ 23248us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 07:08:19.619646 (+ 2394us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44752: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 07:08:19.619921 5944 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44752: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.156.126:41585: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 07:08:19.620641 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 5859
2026-08-12T07:08:19Z chronyd exiting
I20260812 07:08:19.664664 32369 test_util.cc:175] -----------------------------------------------
I20260812 07:08:19.664776 32369 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518468557119-32369-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (402 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 07:08:19.666220 32369 mini_oidc.cc:152] Starting JWKS server
I20260812 07:08:19.667299 32369 webserver.cc:460] Webserver started at https://127.0.0.1:38113/ using document root <none> and password file <none>
I20260812 07:08:19.667395 32369 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 07:08:19.667631 32369 webserver.cc:460] Webserver started at http://127.0.0.1:37499/ using document root <none> and password file <none>
2026-08-12T07:08:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:19Z Disabled control of system clock
I20260812 07:08:19.681608 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:43695
--webserver_interface=127.31.156.126
--webserver_port=0
--builtin_ntp_servers=127.31.156.84:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:43695
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38113/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/testchainca.cert with env {}
W20260812 07:08:19.810302 5984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:19.810684 5984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:19.810799 5984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:19.815011 5984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:19.815150 5984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:19.815181 5984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:19.815227 5984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:19.815294 5984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:08:19.815338 5984 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:08:19.815369 5984 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38113/jwks.json
I20260812 07:08:19.821539 5984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:43695
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:43695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38113/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:19.822863 5984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:19.824074 5984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:19.834458 5990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:19.834517 5994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:19.834782 5984 server_base.cc:1034] running on GCE node
W20260812 07:08:19.834457 5991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:19.835529 5984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:19.836834 5984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:19.838022 5984 hybrid_clock.cc:648] HybridClock initialized: now 1786518499837992 us; error 47 us; skew 500 ppm
I20260812 07:08:19.840440 5984 webserver.cc:460] Webserver started at http://127.31.156.126:33961/ using document root <none> and password file <none>
I20260812 07:08:19.841120 5984 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:19.841192 5984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:19.841428 5984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:19.843968 5984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "d840a420212d49ce8f2e74111c6b72b9"
format_stamp: "Formatted at 2026-08-12 07:08:19 on dist-test-slave-csg5"
server_key: "70f99637aff173040c2985966fa9d3a6"
server_key_iv: "959f11c6358b2f87e499a3db2b81526a"
server_key_version: "encryptionkey@0"
I20260812 07:08:19.844615 5984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "d840a420212d49ce8f2e74111c6b72b9"
format_stamp: "Formatted at 2026-08-12 07:08:19 on dist-test-slave-csg5"
server_key: "70f99637aff173040c2985966fa9d3a6"
server_key_iv: "959f11c6358b2f87e499a3db2b81526a"
server_key_version: "encryptionkey@0"
I20260812 07:08:19.849006 5984 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:19.852088 6001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:19.853806 5984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:08:19.853981 5984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "d840a420212d49ce8f2e74111c6b72b9"
format_stamp: "Formatted at 2026-08-12 07:08:19 on dist-test-slave-csg5"
server_key: "70f99637aff173040c2985966fa9d3a6"
server_key_iv: "959f11c6358b2f87e499a3db2b81526a"
server_key_version: "encryptionkey@0"
I20260812 07:08:19.854143 5984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:19.878961 5984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:19.879798 5984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:19.880039 5984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:19.888939 5984 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:43695
I20260812 07:08:19.889111 6064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:43695 every 8 connection(s)
I20260812 07:08:19.890148 5984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:19.890363 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 5984
I20260812 07:08:19.890520 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:19.890908 32369 external_mini_cluster.cc:1428] Setting key 5ad3bc1d85db592e2603afbc4583f98c
I20260812 07:08:19.893996 6065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:19.901681 6065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9: Bootstrap starting.
I20260812 07:08:19.902000 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.892523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34622 (local address 127.31.156.126:43695)
0812 07:08:19.893246 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:19.893263 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:19.893302 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:08:19.893994 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:19.894013 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:19.894523 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:19.894907 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:19.895294 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.896358 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.897725 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.898290 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.898441 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:19.899551 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:19.899557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:19.899572 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:19.900132 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:19.900317 (+ 185us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:19.900350 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:19.900455 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:19.900747 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 07:08:19.901112 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":170,"threads_started":1}
I20260812 07:08:19.904726 6065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:19.905737 6065 log.cc:824] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:19.905956 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:08:19.908483 6065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9: No bootstrap required, opened a new log
I20260812 07:08:19.912695 6065 raft_consensus.cc:348] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d840a420212d49ce8f2e74111c6b72b9" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43695 } }
I20260812 07:08:19.913020 6065 raft_consensus.cc:374] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:19.913112 6065 raft_consensus.cc:729] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d840a420212d49ce8f2e74111c6b72b9, State: Initialized, Role: FOLLOWER
I20260812 07:08:19.913661 6065 consensus_queue.cc:260] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d840a420212d49ce8f2e74111c6b72b9" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43695 } }
I20260812 07:08:19.913885 6070 raft_consensus.cc:482] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:19.914023 6070 raft_consensus.cc:504] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d840a420212d49ce8f2e74111c6b72b9" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43695 } }
I20260812 07:08:19.913856 6065 raft_consensus.cc:388] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:19.914397 6065 raft_consensus.cc:482] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:19.914391 6070 leader_election.cc:302] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d840a420212d49ce8f2e74111c6b72b9; no voters:
I20260812 07:08:19.914479 6065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:19.914865 6070 leader_election.cc:288] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260812 07:08:19.915884 6065 raft_consensus.cc:504] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d840a420212d49ce8f2e74111c6b72b9" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43695 } }
I20260812 07:08:19.916005 6065 leader_election.cc:302] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d840a420212d49ce8f2e74111c6b72b9; no voters:
I20260812 07:08:19.916085 6071 raft_consensus.cc:2741] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260812 07:08:19.916108 6065 leader_election.cc:288] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:19.916477 6070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:19.916749 6070 raft_consensus.cc:686] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [term 1 LEADER]: Becoming Leader. State: Replica: d840a420212d49ce8f2e74111c6b72b9, State: Running, Role: LEADER
I20260812 07:08:19.917114 6070 consensus_queue.cc:237] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d840a420212d49ce8f2e74111c6b72b9" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43695 } }
I20260812 07:08:19.917743 6065 sys_catalog.cc:564] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:19.918830 6072 sys_catalog.cc:455] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d840a420212d49ce8f2e74111c6b72b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d840a420212d49ce8f2e74111c6b72b9" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43695 } } }
I20260812 07:08:19.918879 6073 sys_catalog.cc:455] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d840a420212d49ce8f2e74111c6b72b9. Latest consensus state: current_term: 1 leader_uuid: "d840a420212d49ce8f2e74111c6b72b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d840a420212d49ce8f2e74111c6b72b9" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43695 } } }
I20260812 07:08:19.918996 6073 sys_catalog.cc:458] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:19.918996 6072 sys_catalog.cc:458] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:19.919602 6082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:19.923009 6082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:19.929469 6082 catalog_manager.cc:1347] Generated new cluster ID: 128d2a43ebfb4b84bb1429c0878c73a4
I20260812 07:08:19.929576 6082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:19.950265 6082 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:19.951594 6082 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:19.958083 6082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d840a420212d49ce8f2e74111c6b72b9: Generated new TSK 0
I20260812 07:08:19.959182 6082 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 07:08:19.988654 6099 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 07:08:19.988768 6099 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 07:08:19.989728 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:19.972702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34632 (local address 127.31.156.126:43695)
0812 07:08:19.972858 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:19.972862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:19.972994 (+ 132us) server_negotiation.cc:407] Connection header received
0812 07:08:19.973229 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:19.973231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:19.973288 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:19.973423 (+ 135us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:08:19.973683 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.974270 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.975292 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.975557 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.975687 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:19.976047 (+ 360us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:08:19.989169 (+ 13122us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 07:08:19.989522 (+ 353us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34632: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 07:08:19.989817 6068 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34632: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 07:08:19.989852 6098 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:19.972519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:43695 (local address 127.0.0.1:34632)
0812 07:08:19.972886 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:19.972901 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:19.973058 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:19.973422 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:19.973426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:19.973434 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:08:19.973561 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:19.973566 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.974407 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.974412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:19.975147 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:19.975157 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:19.975703 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:19.975708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:19.975804 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:19.975831 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:08:19.989419 (+ 13588us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 07:08:19.989646 (+ 227us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.156.126:43695: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":120,"threads_started":1}
I20260812 07:08:19.990622 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 5984
2026-08-12T07:08:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (370 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 07:08:20.036671 32369 mini_oidc.cc:152] Starting JWKS server
I20260812 07:08:20.037621 32369 webserver.cc:460] Webserver started at https://127.0.0.1:38123/ using document root <none> and password file <none>
I20260812 07:08:20.037741 32369 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 07:08:20.037976 32369 webserver.cc:460] Webserver started at http://127.0.0.1:32785/ using document root <none> and password file <none>
2026-08-12T07:08:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:20Z Disabled control of system clock
I20260812 07:08:20.050122 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:36305
--webserver_interface=127.31.156.126
--webserver_port=0
--builtin_ntp_servers=127.31.156.84:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:36305
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38123/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 07:08:20.179140 6106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:20.179528 6106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:20.179575 6106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:20.183892 6106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:20.183975 6106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:20.183991 6106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:20.184008 6106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:20.184026 6106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:08:20.184042 6106 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:08:20.184055 6106 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38123/jwks.json
I20260812 07:08:20.189272 6106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:36305
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:36305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38123/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:20.190582 6106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:20.192354 6106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:20.203208 6111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:20.203209 6114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:20.203258 6112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:20.203593 6106 server_base.cc:1034] running on GCE node
I20260812 07:08:20.204476 6106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:20.205819 6106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:20.207015 6106 hybrid_clock.cc:648] HybridClock initialized: now 1786518500206992 us; error 34 us; skew 500 ppm
I20260812 07:08:20.209570 6106 webserver.cc:460] Webserver started at http://127.31.156.126:38535/ using document root <none> and password file <none>
I20260812 07:08:20.210253 6106 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:20.210341 6106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:20.210567 6106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:20.213243 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "6602899ff6a3426a806c66295cada089"
format_stamp: "Formatted at 2026-08-12 07:08:20 on dist-test-slave-csg5"
server_key: "12909dfa298baf9511c19422142d0320"
server_key_iv: "e4bcd0b876f4c99a3e6a091f556e4358"
server_key_version: "encryptionkey@0"
I20260812 07:08:20.213904 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "6602899ff6a3426a806c66295cada089"
format_stamp: "Formatted at 2026-08-12 07:08:20 on dist-test-slave-csg5"
server_key: "12909dfa298baf9511c19422142d0320"
server_key_iv: "e4bcd0b876f4c99a3e6a091f556e4358"
server_key_version: "encryptionkey@0"
I20260812 07:08:20.219046 6106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 07:08:20.222337 6120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:20.223915 6106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:20.224198 6106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "6602899ff6a3426a806c66295cada089"
format_stamp: "Formatted at 2026-08-12 07:08:20 on dist-test-slave-csg5"
server_key: "12909dfa298baf9511c19422142d0320"
server_key_iv: "e4bcd0b876f4c99a3e6a091f556e4358"
server_key_version: "encryptionkey@0"
I20260812 07:08:20.224356 6106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:20.243216 6106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:20.244206 6106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:20.244467 6106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:20.253187 6106 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:36305
I20260812 07:08:20.253196 6182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:36305 every 8 connection(s)
I20260812 07:08:20.254357 6106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:20.258399 6183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:20.258513 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 6106
I20260812 07:08:20.258709 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:20.259022 32369 external_mini_cluster.cc:1428] Setting key 38bab7d003a185bf3bebbe083e07290a
I20260812 07:08:20.266577 6183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089: Bootstrap starting.
I20260812 07:08:20.269690 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:20.260593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38900 (local address 127.31.156.126:36305)
0812 07:08:20.261323 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:20.261341 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:20.261387 (+ 46us) server_negotiation.cc:407] Connection header received
0812 07:08:20.262109 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:20.262130 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:20.262578 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:20.262971 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:20.263342 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.264367 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.265663 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.266205 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.266354 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:20.267538 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:20.267544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:20.267558 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:20.268027 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:20.268095 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:20.268118 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:20.268210 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:20.268469 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 07:08:20.268817 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":179,"threads_started":1}
I20260812 07:08:20.269795 6183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:20.270613 6183 log.cc:824] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:20.273099 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:08:20.273137 6183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089: No bootstrap required, opened a new log
I20260812 07:08:20.277612 6183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6602899ff6a3426a806c66295cada089" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36305 } }
I20260812 07:08:20.277894 6183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:20.277932 6183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6602899ff6a3426a806c66295cada089, State: Initialized, Role: FOLLOWER
I20260812 07:08:20.278378 6183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6602899ff6a3426a806c66295cada089" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36305 } }
I20260812 07:08:20.278512 6183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:20.278558 6183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:20.278640 6183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:20.279825 6183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6602899ff6a3426a806c66295cada089" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36305 } }
I20260812 07:08:20.280182 6183 leader_election.cc:302] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6602899ff6a3426a806c66295cada089; no voters:
I20260812 07:08:20.280500 6183 leader_election.cc:288] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:20.280794 6189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:20.281049 6189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [term 1 LEADER]: Becoming Leader. State: Replica: 6602899ff6a3426a806c66295cada089, State: Running, Role: LEADER
I20260812 07:08:20.281395 6189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6602899ff6a3426a806c66295cada089" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36305 } }
I20260812 07:08:20.282019 6183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:20.283385 6190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6602899ff6a3426a806c66295cada089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6602899ff6a3426a806c66295cada089" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36305 } } }
I20260812 07:08:20.283371 6191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6602899ff6a3426a806c66295cada089. Latest consensus state: current_term: 1 leader_uuid: "6602899ff6a3426a806c66295cada089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6602899ff6a3426a806c66295cada089" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36305 } } }
I20260812 07:08:20.283542 6190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:20.283542 6191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:20.284188 6204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:20.287758 6204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:20.294777 6204 catalog_manager.cc:1347] Generated new cluster ID: e038f4e0404e4b7ea925a0674aed3259
I20260812 07:08:20.294885 6204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:20.315215 6204 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:20.316711 6204 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:20.329329 6204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6602899ff6a3426a806c66295cada089: Generated new TSK 0
I20260812 07:08:20.330188 6204 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:20.359865 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:20.349449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38908 (local address 127.31.156.126:36305)
0812 07:08:20.349649 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:20.349657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:20.351112 (+ 1455us) server_negotiation.cc:407] Connection header received
0812 07:08:20.351354 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:20.351359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:20.351465 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:20.351654 (+ 189us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:08:20.351978 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.353023 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.354434 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.354771 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.354907 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:20.355346 (+ 439us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:08:20.359375 (+ 4029us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 07:08:20.359702 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38908: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 07:08:20.359970 6187 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38908: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 07:08:20.363023 6219 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:20.349377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:36305 (local address 127.0.0.1:38908)
0812 07:08:20.350946 (+ 1569us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:20.350967 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:20.351139 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:20.351654 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:20.351659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:20.351671 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:08:20.351856 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:20.351866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.353222 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.353226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:20.354273 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:20.354291 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.354939 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.354943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:20.355041 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:20.355069 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:08:20.359592 (+ 4523us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 07:08:20.362577 (+ 2985us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.156.126:36305: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38123/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":1447,"mutex_wait_us":2712,"thread_start_us":123,"threads_started":1}
I20260812 07:08:20.363693 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 6106
E20260812 07:08:20.375304 6220 webserver.cc:533] Webserver: sslize failed:
E20260812 07:08:20.375402 6220 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T07:08:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (372 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T07:08:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:20Z Disabled control of system clock
I20260812 07:08:20.423132 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:43657
--webserver_interface=127.31.156.126
--webserver_port=0
--builtin_ntp_servers=127.31.156.84:35365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:43657
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/testchainca.cert with env {}
W20260812 07:08:20.559487 6224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:20.559824 6224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:20.559870 6224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:20.564198 6224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:20.564284 6224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:20.564301 6224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:20.564318 6224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:20.564334 6224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:08:20.564352 6224 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:08:20.564364 6224 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 07:08:20.570292 6224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:35365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:43657
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:43657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:20.571660 6224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:20.572768 6224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:20.583031 6231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:20.583020 6230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:20.583072 6233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:20.583104 6224 server_base.cc:1034] running on GCE node
I20260812 07:08:20.584163 6224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:20.585565 6224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:20.586783 6224 hybrid_clock.cc:648] HybridClock initialized: now 1786518500586751 us; error 40 us; skew 500 ppm
I20260812 07:08:20.589252 6224 webserver.cc:460] Webserver started at http://127.31.156.126:35407/ using document root <none> and password file <none>
I20260812 07:08:20.589936 6224 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:20.590052 6224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:20.590358 6224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:20.592913 6224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "97954375d64843b98ad13cc0acd81134"
format_stamp: "Formatted at 2026-08-12 07:08:20 on dist-test-slave-csg5"
server_key: "7d84430bf760fff14e34e0119ef75a7d"
server_key_iv: "a4f8e35da17130103214e4c178ebe555"
server_key_version: "encryptionkey@0"
I20260812 07:08:20.593600 6224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "97954375d64843b98ad13cc0acd81134"
format_stamp: "Formatted at 2026-08-12 07:08:20 on dist-test-slave-csg5"
server_key: "7d84430bf760fff14e34e0119ef75a7d"
server_key_iv: "a4f8e35da17130103214e4c178ebe555"
server_key_version: "encryptionkey@0"
I20260812 07:08:20.598196 6224 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 07:08:20.601434 6239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:20.603075 6224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:20.603343 6224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "97954375d64843b98ad13cc0acd81134"
format_stamp: "Formatted at 2026-08-12 07:08:20 on dist-test-slave-csg5"
server_key: "7d84430bf760fff14e34e0119ef75a7d"
server_key_iv: "a4f8e35da17130103214e4c178ebe555"
server_key_version: "encryptionkey@0"
I20260812 07:08:20.603510 6224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:20.618093 6224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:20.619056 6224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:20.619320 6224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:20.628068 6224 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:43657
I20260812 07:08:20.628109 6298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:43657 every 8 connection(s)
I20260812 07:08:20.629253 6224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:20.632699 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 6224
I20260812 07:08:20.632884 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:20.632885 6299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:20.633374 32369 external_mini_cluster.cc:1428] Setting key 57ae6921dd4ad5db641eca3bb4dd7057
I20260812 07:08:20.640647 6299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134: Bootstrap starting.
I20260812 07:08:20.643529 6299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:20.644433 6299 log.cc:824] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:20.644744 6302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:20.635015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42528 (local address 127.31.156.126:43657)
0812 07:08:20.635952 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:20.635970 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:20.636026 (+ 56us) server_negotiation.cc:407] Connection header received
0812 07:08:20.636806 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:20.636828 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:20.637281 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:20.637731 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:20.638227 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.639430 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.640519 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.641142 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.641340 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:20.642532 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:20.642538 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:20.642552 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:20.643071 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:20.643138 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:20.643161 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:20.643348 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:20.643526 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 07:08:20.643856 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":197,"threads_started":1}
I20260812 07:08:20.647183 6299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134: No bootstrap required, opened a new log
I20260812 07:08:20.648553 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:08:20.652683 6299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97954375d64843b98ad13cc0acd81134" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43657 } }
I20260812 07:08:20.653033 6299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:20.653134 6299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97954375d64843b98ad13cc0acd81134, State: Initialized, Role: FOLLOWER
I20260812 07:08:20.653704 6299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97954375d64843b98ad13cc0acd81134" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43657 } }
I20260812 07:08:20.653900 6299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:20.654006 6299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:20.654136 6299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:20.655429 6299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97954375d64843b98ad13cc0acd81134" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43657 } }
I20260812 07:08:20.655864 6299 leader_election.cc:302] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97954375d64843b98ad13cc0acd81134; no voters:
I20260812 07:08:20.656265 6299 leader_election.cc:288] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:20.656473 6304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:20.656941 6304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [term 1 LEADER]: Becoming Leader. State: Replica: 97954375d64843b98ad13cc0acd81134, State: Running, Role: LEADER
I20260812 07:08:20.657315 6304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97954375d64843b98ad13cc0acd81134" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43657 } }
I20260812 07:08:20.657862 6299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:20.659611 6305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97954375d64843b98ad13cc0acd81134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97954375d64843b98ad13cc0acd81134" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43657 } } }
I20260812 07:08:20.659765 6305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:20.659574 6306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97954375d64843b98ad13cc0acd81134. Latest consensus state: current_term: 1 leader_uuid: "97954375d64843b98ad13cc0acd81134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97954375d64843b98ad13cc0acd81134" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43657 } } }
I20260812 07:08:20.659895 6306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:20.661103 6315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:20.664503 6315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:20.671249 6315 catalog_manager.cc:1347] Generated new cluster ID: 497d2201eb0a449e86346d1da9fd2ef9
I20260812 07:08:20.671348 6315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:20.689105 6315 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:20.690274 6315 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:20.697482 6315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97954375d64843b98ad13cc0acd81134: Generated new TSK 0
I20260812 07:08:20.698377 6315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:21.138454 6302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:20.717213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42534 (local address 127.31.156.126:43657)
0812 07:08:20.717485 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:20.717491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:20.717586 (+ 95us) server_negotiation.cc:407] Connection header received
0812 07:08:20.717810 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:20.717814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:20.717898 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:20.718048 (+ 150us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:08:20.718376 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.719407 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.720594 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.721006 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.721174 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:20.721673 (+ 499us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:08:21.137960 (+416287us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 07:08:21.138311 (+ 351us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42534: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":136}
W20260812 07:08:21.138540 6302 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42534: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 07:08:21.138834 6332 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:20.717054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:43657 (local address 127.0.0.1:42534)
0812 07:08:20.717463 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:20.717482 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:20.717609 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:20.718088 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:20.718091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:20.718102 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:08:20.718224 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:20.718230 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.719590 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.719593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:20.720417 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:20.720430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:20.721287 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:20.721292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:20.721383 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:20.721418 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:08:21.138292 (+416874us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 07:08:21.138514 (+ 222us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.156.126:43657: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":152,"threads_started":1}
I20260812 07:08:21.139739 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 6224
2026-08-12T07:08:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (754 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (148 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (162 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:21 dist-test-slave-csg5 krb5kdc[6348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:21 dist-test-slave-csg5 krb5kdc[6348](info): set up 2 sockets
Aug 12 07:08:21 dist-test-slave-csg5 krb5kdc[6348](info): commencing operation
krb5kdc: starting...
W20260812 07:08:22.619300 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:08:22 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:08:22.783315 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:36833
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:36833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:22.905947 6363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:22.906303 6363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:22.906348 6363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:22.910745 6363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:22.910833 6363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:22.910854 6363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:22.910871 6363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:22.910888 6363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:22.916904 6363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:36833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:36833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:22.918298 6363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:22.919513 6363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:22.929160 6372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:22.929292 6369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:22.929247 6363 server_base.cc:1034] running on GCE node
W20260812 07:08:22.929170 6368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:22.930056 6363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:22.931463 6363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:22.932735 6363 hybrid_clock.cc:648] HybridClock initialized: now 1786518502932673 us; error 69 us; skew 500 ppm
Aug 12 07:08:22 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518502, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:22.941246 6363 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:08:22.942387 6363 webserver.cc:460] Webserver started at http://127.31.156.126:40105/ using document root <none> and password file <none>
I20260812 07:08:22.943050 6363 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:22.943122 6363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:22.943315 6363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:22.945544 6363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "9f0fc7713e46480a817fe6a924a9a775"
format_stamp: "Formatted at 2026-08-12 07:08:22 on dist-test-slave-csg5"
server_key: "73a4fa32d0a4a103b9d040a6c7d9830c"
server_key_iv: "d4eaaed81742fb386496dac9d3d8f4c8"
server_key_version: "encryptionkey@0"
I20260812 07:08:22.946115 6363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "9f0fc7713e46480a817fe6a924a9a775"
format_stamp: "Formatted at 2026-08-12 07:08:22 on dist-test-slave-csg5"
server_key: "73a4fa32d0a4a103b9d040a6c7d9830c"
server_key_iv: "d4eaaed81742fb386496dac9d3d8f4c8"
server_key_version: "encryptionkey@0"
I20260812 07:08:22.950402 6363 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260812 07:08:22.953727 6380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:22.955497 6363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:08:22.955672 6363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "9f0fc7713e46480a817fe6a924a9a775"
format_stamp: "Formatted at 2026-08-12 07:08:22 on dist-test-slave-csg5"
server_key: "73a4fa32d0a4a103b9d040a6c7d9830c"
server_key_iv: "d4eaaed81742fb386496dac9d3d8f4c8"
server_key_version: "encryptionkey@0"
I20260812 07:08:22.955811 6363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:22.974597 6363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:22.979657 6363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:22.980015 6363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:22.988674 6363 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:36833
I20260812 07:08:22.988775 6436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:36833 every 8 connection(s)
I20260812 07:08:22.989876 6363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:22.992763 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 6363
I20260812 07:08:22.992949 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:22.993248 32369 external_mini_cluster.cc:1428] Setting key 598ed018fa8e8b2993fa6a8cedf3a926
I20260812 07:08:22.993404 6437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:23.001571 6437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775: Bootstrap starting.
Aug 12 07:08:22 dist-test-slave-csg5 krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:23.004799 6437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:23.005671 6437 log.cc:824] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:23.007983 6437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775: No bootstrap required, opened a new log
I20260812 07:08:23.010013 6440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:22.994911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59700 (local address 127.31.156.126:36833)
0812 07:08:22.995349 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:22.995372 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:22.995410 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:08:22.996479 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:22.996500 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:22.996875 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:22.997184 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:22.998316 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:22.999506 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.000750 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.001257 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.001438 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:23.004707 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:23.004719 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:23.004734 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:23.004772 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:23.006969 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.007480 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.007491 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.007496 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.007613 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.007928 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.007936 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.007938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.008402 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:23.008612 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:23.008933 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 07:08:23.009249 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":155,"threads_started":1}
I20260812 07:08:23.012243 6437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0fc7713e46480a817fe6a924a9a775" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36833 } }
I20260812 07:08:23.012570 6437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:23.012640 6437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0fc7713e46480a817fe6a924a9a775, State: Initialized, Role: FOLLOWER
I20260812 07:08:23.013077 6437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0fc7713e46480a817fe6a924a9a775" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36833 } }
I20260812 07:08:23.013207 6437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:23.013249 6437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:23.013317 6437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:23.014344 6437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0fc7713e46480a817fe6a924a9a775" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36833 } }
I20260812 07:08:23.014762 6437 leader_election.cc:302] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f0fc7713e46480a817fe6a924a9a775; no voters:
I20260812 07:08:23.015111 6437 leader_election.cc:288] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:23.015286 6442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:23.015573 6442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [term 1 LEADER]: Becoming Leader. State: Replica: 9f0fc7713e46480a817fe6a924a9a775, State: Running, Role: LEADER
I20260812 07:08:23.015899 6442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0fc7713e46480a817fe6a924a9a775" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36833 } }
I20260812 07:08:23.016513 6437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:23.017717 6444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f0fc7713e46480a817fe6a924a9a775. Latest consensus state: current_term: 1 leader_uuid: "9f0fc7713e46480a817fe6a924a9a775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0fc7713e46480a817fe6a924a9a775" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36833 } } }
I20260812 07:08:23.017858 6444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:23.018316 6451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:23.019316 6443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f0fc7713e46480a817fe6a924a9a775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0fc7713e46480a817fe6a924a9a775" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36833 } } }
I20260812 07:08:23.019433 6443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:23.021334 6451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:23.028295 6451 catalog_manager.cc:1347] Generated new cluster ID: 5a47a69f79914e638ec33e3c756bf92e
I20260812 07:08:23.028393 6451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:23.051342 6451 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:23.052775 6451 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:23.066860 6451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f0fc7713e46480a817fe6a924a9a775: Generated new TSK 0
I20260812 07:08:23.067764 6451 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:08:23.112147 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36833
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:23.239925 6464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:23.240242 6464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:23.240288 6464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:23.244757 6464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:23.244845 6464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:23.244910 6464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:23.250846 6464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:23.252146 6464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:23.253429 6464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:23.265681 6472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:23.265794 6464 server_base.cc:1034] running on GCE node
W20260812 07:08:23.265672 6475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:23.265681 6471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:23.266703 6464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:23.267685 6464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:23.268996 6464 hybrid_clock.cc:648] HybridClock initialized: now 1786518503268881 us; error 135 us; skew 500 ppm
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:23.273036 6464 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:08:23.274281 6464 webserver.cc:460] Webserver started at http://127.31.156.65:35083/ using document root <none> and password file <none>
I20260812 07:08:23.275130 6464 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:23.275249 6464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:23.275530 6464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:23.278132 6464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "0e29d31e36f14de7bb8fe0bfd062c262"
format_stamp: "Formatted at 2026-08-12 07:08:23 on dist-test-slave-csg5"
server_key: "96ab2cbb90c29d53e5ef184cd487f04d"
server_key_iv: "48c72b9d3ef29d54fbab3480953ad2d4"
server_key_version: "encryptionkey@0"
I20260812 07:08:23.278909 6464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "0e29d31e36f14de7bb8fe0bfd062c262"
format_stamp: "Formatted at 2026-08-12 07:08:23 on dist-test-slave-csg5"
server_key: "96ab2cbb90c29d53e5ef184cd487f04d"
server_key_iv: "48c72b9d3ef29d54fbab3480953ad2d4"
server_key_version: "encryptionkey@0"
I20260812 07:08:23.284138 6464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 07:08:23.288051 6484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:23.289772 6464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:08:23.289973 6464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "0e29d31e36f14de7bb8fe0bfd062c262"
format_stamp: "Formatted at 2026-08-12 07:08:23 on dist-test-slave-csg5"
server_key: "96ab2cbb90c29d53e5ef184cd487f04d"
server_key_iv: "48c72b9d3ef29d54fbab3480953ad2d4"
server_key_version: "encryptionkey@0"
I20260812 07:08:23.290140 6464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:23.303988 6464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:23.307912 6464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:23.308166 6464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:23.308866 6464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:23.309845 6464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:23.309929 6464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:23.310006 6464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:23.310045 6464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:23.323038 6464 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:46605
I20260812 07:08:23.323086 6612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:46605 every 8 connection(s)
I20260812 07:08:23.324110 6464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:23.331816 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 6464
I20260812 07:08:23.331969 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:08:23.332316 32369 external_mini_cluster.cc:1428] Setting key bc810691bae8b779cfc53266feadda67
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:23.343760 6440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:23.326636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:57501 (local address 127.31.156.126:36833)
0812 07:08:23.326846 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:23.326851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:23.327931 (+ 1080us) server_negotiation.cc:407] Connection header received
0812 07:08:23.329536 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:23.329540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:23.329613 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:23.329718 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:23.331200 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.331971 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.333742 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.334134 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.334279 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:23.338141 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:23.338145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:23.338147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:23.338179 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:23.340406 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.341312 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.341319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.341323 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.341454 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.342021 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.342027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.342031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.342379 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:23.342519 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:23.343319 (+ 800us) server_negotiation.cc:299] Negotiation successful
0812 07:08:23.343514 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260812 07:08:23.344712 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:23.327102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:36833 (local address 127.31.156.65:57501)
0812 07:08:23.327705 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:23.327763 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:23.329254 (+ 1491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:23.329970 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:23.329979 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:23.330432 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:23.331010 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:23.331024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.332164 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.332168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:23.333555 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:23.333570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.334370 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.334375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:23.334616 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:23.335470 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:23.335503 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:23.337868 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:23.340600 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:23.340663 (+ 63us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:23.340681 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:23.341128 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:23.341593 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:23.341596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:23.341598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:23.341860 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:23.342547 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:23.342555 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:23.342985 (+ 430us) client_negotiation.cc:769] Sending connection context
0812 07:08:23.343202 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 07:08:23.343549 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":153,"threads_started":1}
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:08:23.346540 6614 heartbeater.cc:347] Connected to a master server at 127.31.156.126:36833
I20260812 07:08:23.346966 6614 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:23.347684 6614 heartbeater.cc:511] Master 127.31.156.126:36833 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
I20260812 07:08:23.349810 6399 ts_manager.cc:194] Registered new tserver with Master: 0e29d31e36f14de7bb8fe0bfd062c262 (127.31.156.65:46605)
I20260812 07:08:23.351666 6399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:57501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:08:23.372035 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36833
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:23.499647 6619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:23.500061 6619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:23.500161 6619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:23.505095 6619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:23.505244 6619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:23.505352 6619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:23.511922 6619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:23.513289 6619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:23.514483 6619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:23.525928 6627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:23.525941 6625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:23.525947 6624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:23.526351 6619 server_base.cc:1034] running on GCE node
I20260812 07:08:23.526888 6619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:23.527719 6619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:23.528959 6619 hybrid_clock.cc:648] HybridClock initialized: now 1786518503528945 us; error 30 us; skew 500 ppm
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:23.532449 6619 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:08:23.533457 6619 webserver.cc:460] Webserver started at http://127.31.156.66:43139/ using document root <none> and password file <none>
I20260812 07:08:23.534147 6619 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:23.534253 6619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:23.534511 6619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:23.536837 6619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "e26b066eff854f828bf08569b09b660f"
format_stamp: "Formatted at 2026-08-12 07:08:23 on dist-test-slave-csg5"
server_key: "5a913f8143949609a2e51e52214dff5f"
server_key_iv: "94895000078dd6bc8b613161d03fad8a"
server_key_version: "encryptionkey@0"
I20260812 07:08:23.537484 6619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "e26b066eff854f828bf08569b09b660f"
format_stamp: "Formatted at 2026-08-12 07:08:23 on dist-test-slave-csg5"
server_key: "5a913f8143949609a2e51e52214dff5f"
server_key_iv: "94895000078dd6bc8b613161d03fad8a"
server_key_version: "encryptionkey@0"
I20260812 07:08:23.541762 6619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:08:23.544652 6634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:23.546116 6619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:23.546360 6619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "e26b066eff854f828bf08569b09b660f"
format_stamp: "Formatted at 2026-08-12 07:08:23 on dist-test-slave-csg5"
server_key: "5a913f8143949609a2e51e52214dff5f"
server_key_iv: "94895000078dd6bc8b613161d03fad8a"
server_key_version: "encryptionkey@0"
I20260812 07:08:23.546509 6619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:23.560233 6619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:23.564049 6619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:23.564297 6619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:23.564967 6619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:23.565969 6619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:23.566064 6619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:23.566138 6619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:23.566187 6619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:23.576608 6619 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:46599
I20260812 07:08:23.576627 6764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:46599 every 8 connection(s)
I20260812 07:08:23.577713 6619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:23.581411 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 6619
I20260812 07:08:23.581552 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:08:23.581931 32369 external_mini_cluster.cc:1428] Setting key 70bb15ab69bebc2388cf34780b67d575
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:08:23.595551 6440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:23.579795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:36911 (local address 127.31.156.126:36833)
0812 07:08:23.579951 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:23.579955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:23.580942 (+ 987us) server_negotiation.cc:407] Connection header received
0812 07:08:23.582977 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:23.582980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:23.583032 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:23.583125 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:23.584876 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.585534 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.587081 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.587470 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.587610 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:23.591413 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:23.591416 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:23.591419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:23.591449 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:23.593046 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.593694 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.593697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.593699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.593785 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.594177 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.594181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.594183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.594399 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:23.594526 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:23.595198 (+ 672us) server_negotiation.cc:299] Negotiation successful
0812 07:08:23.595353 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:08:23.596537 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:23.580240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:36833 (local address 127.31.156.66:36911)
0812 07:08:23.580761 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:23.580810 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:23.582715 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:23.583360 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:23.583370 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:23.583860 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:23.584650 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:23.584678 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.585696 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.585703 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:23.586944 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:23.586955 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.587783 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.587790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:23.588073 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:23.588872 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:23.588911 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:23.591185 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:23.593196 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:23.593203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:23.593222 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:23.593552 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:23.593888 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:23.593892 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:23.593894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:23.594057 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:23.594526 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:23.594535 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:23.594913 (+ 378us) client_negotiation.cc:769] Sending connection context
0812 07:08:23.595127 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 07:08:23.595514 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":116,"threads_started":1}
I20260812 07:08:23.597990 6765 heartbeater.cc:347] Connected to a master server at 127.31.156.126:36833
I20260812 07:08:23.598371 6765 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
I20260812 07:08:23.599049 6765 heartbeater.cc:511] Master 127.31.156.126:36833 requested a full tablet report, sending...
I20260812 07:08:23.600455 6399 ts_manager.cc:194] Registered new tserver with Master: e26b066eff854f828bf08569b09b660f (127.31.156.66:46599)
I20260812 07:08:23.601399 6399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:36911
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:08:23.619589 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36833
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:23.763118 6772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:23.763453 6772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:23.763504 6772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:23.768203 6772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:23.768289 6772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:23.768360 6772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:23.774304 6772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:23.775540 6772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:23.776589 6772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:23.787745 6780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:23.787858 6772 server_base.cc:1034] running on GCE node
W20260812 07:08:23.787771 6779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:23.787750 6782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:23.788571 6772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:23.789390 6772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:23.790606 6772 hybrid_clock.cc:648] HybridClock initialized: now 1786518503790588 us; error 36 us; skew 500 ppm
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:23.794301 6772 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:08:23.795595 6772 webserver.cc:460] Webserver started at http://127.31.156.67:38035/ using document root <none> and password file <none>
I20260812 07:08:23.796190 6772 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:23.796263 6772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:23.796469 6772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:23.798800 6772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "1fd8dfe6b90341d58f885b31e49ca6da"
format_stamp: "Formatted at 2026-08-12 07:08:23 on dist-test-slave-csg5"
server_key: "dfa9a6f118c2ab376bbb8f7b294cb5be"
server_key_iv: "64c683814ec2ad4d9083d67b00d49dee"
server_key_version: "encryptionkey@0"
I20260812 07:08:23.799424 6772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "1fd8dfe6b90341d58f885b31e49ca6da"
format_stamp: "Formatted at 2026-08-12 07:08:23 on dist-test-slave-csg5"
server_key: "dfa9a6f118c2ab376bbb8f7b294cb5be"
server_key_iv: "64c683814ec2ad4d9083d67b00d49dee"
server_key_version: "encryptionkey@0"
I20260812 07:08:23.803759 6772 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:08:23.806495 6791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:23.808243 6772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:08:23.808379 6772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "1fd8dfe6b90341d58f885b31e49ca6da"
format_stamp: "Formatted at 2026-08-12 07:08:23 on dist-test-slave-csg5"
server_key: "dfa9a6f118c2ab376bbb8f7b294cb5be"
server_key_iv: "64c683814ec2ad4d9083d67b00d49dee"
server_key_version: "encryptionkey@0"
I20260812 07:08:23.808562 6772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:23.831153 6772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:23.835028 6772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:23.835239 6772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:23.835873 6772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:23.836777 6772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:23.836833 6772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:23.836879 6772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:23.836896 6772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:23.847800 6772 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:34795
I20260812 07:08:23.847923 6916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:34795 every 8 connection(s)
I20260812 07:08:23.848896 6772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:23.850003 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 6772
I20260812 07:08:23.850148 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:08:23.850529 32369 external_mini_cluster.cc:1428] Setting key f5838cdb32e8811d4191a55103669f94
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:23.867710 6440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:23.852697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:53495 (local address 127.31.156.126:36833)
0812 07:08:23.852871 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:23.852876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:23.853972 (+ 1096us) server_negotiation.cc:407] Connection header received
0812 07:08:23.855601 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:23.855605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:23.855666 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:23.855748 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:23.857087 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.857657 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.859033 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.859278 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.859403 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:23.862898 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:23.862902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:23.862904 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:23.862936 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:23.864988 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.865765 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.865770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.865777 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.865869 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.866332 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.866336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.866338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.866566 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:23.866724 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:23.867365 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 07:08:23.867522 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 07:08:23.868671 6921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:23.853190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:36833 (local address 127.31.156.67:53495)
0812 07:08:23.853813 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:23.853863 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:23.855361 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:23.855968 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:23.855976 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:23.856381 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:23.856916 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:23.856931 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.857815 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.857819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:23.858893 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:23.858906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.859474 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.859478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:23.859701 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:23.860462 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:23.860483 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:23.862557 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:23.865161 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:23.865169 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:23.865194 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:23.865618 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:23.865992 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:23.865995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:23.865997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:23.866197 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:23.866729 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:23.866739 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:23.867068 (+ 329us) client_negotiation.cc:769] Sending connection context
0812 07:08:23.867297 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 07:08:23.867699 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":193,"threads_started":1}
I20260812 07:08:23.870191 6917 heartbeater.cc:347] Connected to a master server at 127.31.156.126:36833
I20260812 07:08:23.870555 6917 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:23.871237 6917 heartbeater.cc:511] Master 127.31.156.126:36833 requested a full tablet report, sending...
I20260812 07:08:23.872506 6399 ts_manager.cc:194] Registered new tserver with Master: 1fd8dfe6b90341d58f885b31e49ca6da (127.31.156.67:34795)
I20260812 07:08:23.873158 6399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:53495
I20260812 07:08:23.878546 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/test-admin.keytab.
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:23.901100 32369 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 07:08:23.904796 6927 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:23.902863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34914 (local address 127.31.156.65:46605)
0812 07:08:23.903265 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:23.903273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:23.903305 (+ 32us) server_negotiation.cc:407] Connection header received
0812 07:08:23.903375 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:23.903380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:23.903574 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:23.903706 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:23.904525 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34914: BlockingRecv error: recv got EOF from 127.0.0.1:34914 (error 108)
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":177,"threads_started":1}
W20260812 07:08:23.904794 32369 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/krb5cc' of type FILE
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:23.906443 32369 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:23.907838 32369 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:08:23.921798 6927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:23.909152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34922 (local address 127.31.156.65:46605)
0812 07:08:23.909326 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:23.909332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:23.909696 (+ 364us) server_negotiation.cc:407] Connection header received
0812 07:08:23.909803 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:23.909806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:23.909859 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:23.909945 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:23.910816 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.911707 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.912625 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.913053 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.913154 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:23.916667 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:23.916690 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:23.916752 (+ 62us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:23.916796 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:23.919454 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.920053 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.920060 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.920061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.920158 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.920557 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.920560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.920561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.920848 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:23.921037 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:23.921387 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 07:08:23.921596 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 07:08:23.926972 6927 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:23.925473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34932 (local address 127.31.156.65:46605)
0812 07:08:23.925641 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:23.925647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:23.925792 (+ 145us) server_negotiation.cc:407] Connection header received
0812 07:08:23.925918 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:23.925920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:23.925979 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:23.926059 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:23.926806 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34932: BlockingRecv error: recv got EOF from 127.0.0.1:34932 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 07:08:23.927021 32369 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/krb5cc' of type FILE
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:23.928452 32369 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:23.929697 32369 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:08:23.941425 6927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:23.930906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34938 (local address 127.31.156.65:46605)
0812 07:08:23.931101 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:23.931105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:23.931171 (+ 66us) server_negotiation.cc:407] Connection header received
0812 07:08:23.931349 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:23.931352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:23.931397 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:23.931475 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:23.932077 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.932668 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.933580 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.933884 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.933977 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:23.936853 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:23.936857 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:23.936859 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:23.936892 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:23.938822 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.939658 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.939662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.939667 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.939823 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.940237 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.940242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.940244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.940520 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:23.940682 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:23.941008 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 07:08:23.941218 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260812 07:08:23.945436 6927 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:23.944221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.31.156.65:46605)
0812 07:08:23.944420 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:23.944423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:23.944499 (+ 76us) server_negotiation.cc:407] Connection header received
0812 07:08:23.944659 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:23.944663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:23.944710 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:23.944793 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:23.945337 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34942: BlockingRecv error: recv got EOF from 127.0.0.1:34942 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260812 07:08:23.945582 32369 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518468557119-32369-0/krb5kdc/krb5cc' of type FILE
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:23.947077 32369 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:23.948359 32369 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 07:08:23 dist-test-slave-csg5 krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:08:23.960413 6927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:23.949845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.31.156.65:46605)
0812 07:08:23.949999 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:23.950003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:23.950022 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:08:23.950356 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:23.950360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:23.950410 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:23.950497 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:23.951593 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.952256 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.953222 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:23.953466 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:23.953562 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:23.956560 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:23.956563 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:23.956566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:23.956595 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:23.958109 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.958592 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.958596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.958597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.958781 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:23.959103 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:23.959106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:23.959108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:23.959430 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:23.959609 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:23.959990 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 07:08:23.960157 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:08:23.961800 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 6464
I20260812 07:08:23.972985 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 6619
I20260812 07:08:23.981415 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 6772
I20260812 07:08:23.994913 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 6363
2026-08-12T07:08:23Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2547 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:24 dist-test-slave-csg5 krb5kdc[6941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:24 dist-test-slave-csg5 krb5kdc[6941](info): set up 2 sockets
Aug 12 07:08:24 dist-test-slave-csg5 krb5kdc[6941](info): commencing operation
krb5kdc: starting...
W20260812 07:08:25.209064 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:08:25 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.126.keytab.
Entry for principal oryx/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.126.keytab.
I20260812 07:08:25.367916 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:41665
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.126.keytab
--principal=oryx/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:41665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:25.492590 6956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:25.492949 6956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:25.492996 6956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:25.497398 6956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:25.497485 6956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:25.497505 6956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:25.497525 6956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:25.497541 6956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:25.503434 6956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:41665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.126.keytab
--principal=oryx/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:41665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:25.504719 6956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:25.505949 6956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:25.516162 6964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:25.516151 6962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:25.516119 6961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:25.516327 6956 server_base.cc:1034] running on GCE node
I20260812 07:08:25.517462 6956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:25.519033 6956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:25.520275 6956 hybrid_clock.cc:648] HybridClock initialized: now 1786518505520237 us; error 65 us; skew 500 ppm
Aug 12 07:08:25 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518505, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:25.523473 6956 init.cc:377] Logged in from keytab as oryx/127.31.156.126@KRBTEST.COM (short username oryx)
I20260812 07:08:25.524528 6956 webserver.cc:460] Webserver started at http://127.31.156.126:39257/ using document root <none> and password file <none>
I20260812 07:08:25.525144 6956 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:25.525247 6956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:25.525527 6956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:25.527877 6956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "0b9ef89962f743399485ce3efb4a26d8"
format_stamp: "Formatted at 2026-08-12 07:08:25 on dist-test-slave-csg5"
server_key: "3b4c9a44e3047ba2362c5c3b7e20a7e1"
server_key_iv: "2361ed3dd093d4bab7d9a2e8debc2cb8"
server_key_version: "encryptionkey@0"
I20260812 07:08:25.528546 6956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "0b9ef89962f743399485ce3efb4a26d8"
format_stamp: "Formatted at 2026-08-12 07:08:25 on dist-test-slave-csg5"
server_key: "3b4c9a44e3047ba2362c5c3b7e20a7e1"
server_key_iv: "2361ed3dd093d4bab7d9a2e8debc2cb8"
server_key_version: "encryptionkey@0"
I20260812 07:08:25.532600 6956 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:25.535557 6972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:25.537014 6956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:25.537179 6956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "0b9ef89962f743399485ce3efb4a26d8"
format_stamp: "Formatted at 2026-08-12 07:08:25 on dist-test-slave-csg5"
server_key: "3b4c9a44e3047ba2362c5c3b7e20a7e1"
server_key_iv: "2361ed3dd093d4bab7d9a2e8debc2cb8"
server_key_version: "encryptionkey@0"
I20260812 07:08:25.537318 6956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:25.553033 6956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:25.557123 6956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:25.557363 6956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:25.566045 6956 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:41665
I20260812 07:08:25.566068 7035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:41665 every 8 connection(s)
I20260812 07:08:25.567257 6956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:25.570261 7036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:25.576315 7036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8: Bootstrap starting.
I20260812 07:08:25.576961 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 6956
I20260812 07:08:25.577104 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:25.577402 32369 external_mini_cluster.cc:1428] Setting key 1166b06ec92e51881c067611540a8dcb
I20260812 07:08:25.579432 7036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:25.580276 7036 log.cc:824] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:25.582429 7036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8: No bootstrap required, opened a new log
I20260812 07:08:25.586376 7036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } }
I20260812 07:08:25.586709 7036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:25.586809 7036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b9ef89962f743399485ce3efb4a26d8, State: Initialized, Role: FOLLOWER
Aug 12 07:08:25 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:25.587352 7036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } }
I20260812 07:08:25.587527 7036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:25.587625 7036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:25.587746 7036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:25.589022 7036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } }
I20260812 07:08:25.589455 7036 leader_election.cc:302] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b9ef89962f743399485ce3efb4a26d8; no voters:
I20260812 07:08:25.589893 7036 leader_election.cc:288] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:25.590036 7041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:25.590379 7041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 1 LEADER]: Becoming Leader. State: Replica: 0b9ef89962f743399485ce3efb4a26d8, State: Running, Role: LEADER
I20260812 07:08:25.590790 7041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } }
I20260812 07:08:25.591542 7036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:25.592857 7043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } }
I20260812 07:08:25.593436 7043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:25.593302 7042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b9ef89962f743399485ce3efb4a26d8. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } }
I20260812 07:08:25.594061 7042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:25.595403 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:25.578983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59700 (local address 127.31.156.126:41665)
0812 07:08:25.579493 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:25.579516 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:25.579578 (+ 62us) server_negotiation.cc:407] Connection header received
0812 07:08:25.580307 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:25.580326 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:25.580754 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:25.581131 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:25.582018 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:25.583255 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:25.584181 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:25.584699 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:25.584884 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:25.588191 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:25.588198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:25.588214 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:25.588249 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:25.590868 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:25.591538 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:25.591544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:25.591552 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:25.591736 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:25.592111 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:25.592114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:25.592116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:25.592740 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:25.592958 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:25.593985 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0812 07:08:25.594325 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":192,"threads_started":1}
W20260812 07:08:25.597079 7056 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:08:25.597183 7056 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:08:25.597288 7055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:25.598137 7055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:25.604836 7055 catalog_manager.cc:1347] Generated new cluster ID: f8f3870a34c84e4791cfae7eac9fbe99
I20260812 07:08:25.604936 7055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:25.627202 7055 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:25.628710 7055 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:25.639256 7055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8: Generated new TSK 0
I20260812 07:08:25.640246 7055 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab.
Entry for principal oryx/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab.
I20260812 07:08:25.680712 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:41665
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab
--principal=oryx/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:25.814752 7063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:25.815064 7063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:25.815111 7063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:25.819463 7063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:25.819545 7063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:25.819607 7063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:25.825657 7063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab
--principal=oryx/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:41665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:25.826977 7063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:25.828125 7063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:25.838763 7074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:25.838773 7069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:25.838815 7072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:25.839274 7063 server_base.cc:1034] running on GCE node
I20260812 07:08:25.839742 7063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:25.841114 7063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:25.842461 7063 hybrid_clock.cc:648] HybridClock initialized: now 1786518505842438 us; error 40 us; skew 500 ppm
Aug 12 07:08:25 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518505, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:25.846009 7063 init.cc:377] Logged in from keytab as oryx/127.31.156.65@KRBTEST.COM (short username oryx)
I20260812 07:08:25.847157 7063 webserver.cc:460] Webserver started at http://127.31.156.65:37231/ using document root <none> and password file <none>
I20260812 07:08:25.847811 7063 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:25.847919 7063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:25.848173 7063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:25.850436 7063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "bfeb604fa3744b96a91e3cca5065e8ba"
format_stamp: "Formatted at 2026-08-12 07:08:25 on dist-test-slave-csg5"
server_key: "6b0ef42dde42be2448f6614d6e84539f"
server_key_iv: "4f78b4120d4aafe5d605891599cf26d9"
server_key_version: "encryptionkey@0"
I20260812 07:08:25.851117 7063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "bfeb604fa3744b96a91e3cca5065e8ba"
format_stamp: "Formatted at 2026-08-12 07:08:25 on dist-test-slave-csg5"
server_key: "6b0ef42dde42be2448f6614d6e84539f"
server_key_iv: "4f78b4120d4aafe5d605891599cf26d9"
server_key_version: "encryptionkey@0"
I20260812 07:08:25.855343 7063 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:08:25.858021 7081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:25.859588 7063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:25.859755 7063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "bfeb604fa3744b96a91e3cca5065e8ba"
format_stamp: "Formatted at 2026-08-12 07:08:25 on dist-test-slave-csg5"
server_key: "6b0ef42dde42be2448f6614d6e84539f"
server_key_iv: "4f78b4120d4aafe5d605891599cf26d9"
server_key_version: "encryptionkey@0"
I20260812 07:08:25.859892 7063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:25.898548 7063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:25.902277 7063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:25.902527 7063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:25.903302 7063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:25.904218 7063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:25.904309 7063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:25.904383 7063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:25.904433 7063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:25.914896 7063 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:38327
I20260812 07:08:25.914952 7213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:38327 every 8 connection(s)
I20260812 07:08:25.915952 7063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:25.920594 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 7063
I20260812 07:08:25.920755 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:08:25.921060 32369 external_mini_cluster.cc:1428] Setting key 4124de07f468940e62dc4b6744ae79b5
Aug 12 07:08:25 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518505, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:25.931418 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:25.918014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:58269 (local address 127.31.156.126:41665)
0812 07:08:25.918199 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:25.918204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:25.919346 (+ 1142us) server_negotiation.cc:407] Connection header received
0812 07:08:25.920834 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:25.920838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:25.920904 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:25.921013 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:25.922521 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:25.923145 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:25.924405 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:25.924664 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:25.924787 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:25.927626 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:25.927629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:25.927631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:25.927663 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:25.929169 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:25.929725 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:25.929728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:25.929730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:25.929817 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:25.930190 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:25.930195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:25.930197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:25.930394 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:25.930509 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:25.931106 (+ 597us) server_negotiation.cc:299] Negotiation successful
0812 07:08:25.931242 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
WARNING: no policy specified for oryx/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:08:25.932322 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:25.918539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.65:58269)
0812 07:08:25.919083 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:25.919127 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:25.920574 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:25.921264 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:25.921271 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:25.921811 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:25.922349 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:25.922365 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:25.923290 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:25.923294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:25.924252 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:25.924263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:25.924839 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:25.924843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:25.925034 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:25.925791 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:25.925809 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:25.927425 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:25.929301 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:25.929308 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:25.929326 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:25.929611 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:25.929916 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:25.929919 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:25.929921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:25.930076 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:25.930509 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:25.930517 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:25.930853 (+ 336us) client_negotiation.cc:769] Sending connection context
0812 07:08:25.931062 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 07:08:25.931433 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":191,"threads_started":1}
I20260812 07:08:25.933914 7214 heartbeater.cc:347] Connected to a master server at 127.31.156.126:41665
I20260812 07:08:25.934304 7214 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:25.934949 7214 heartbeater.cc:511] Master 127.31.156.126:41665 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.66@KRBTEST.COM" created.
I20260812 07:08:25.937050 6992 ts_manager.cc:194] Registered new tserver with Master: bfeb604fa3744b96a91e3cca5065e8ba (127.31.156.65:38327)
I20260812 07:08:25.938437 6992 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:58269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab.
Entry for principal oryx/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab.
I20260812 07:08:25.960106 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:41665
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab
--principal=oryx/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:26.086366 7221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:26.086777 7221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:26.086850 7221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:26.091374 7221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:26.091452 7221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:26.091517 7221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:26.097779 7221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab
--principal=oryx/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:41665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:26.099330 7221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:26.100387 7221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:26.111097 7228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:26.111091 7230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:26.111092 7227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:26.111150 7221 server_base.cc:1034] running on GCE node
I20260812 07:08:26.112041 7221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:26.112767 7221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:26.113969 7221 hybrid_clock.cc:648] HybridClock initialized: now 1786518506113925 us; error 48 us; skew 500 ppm
Aug 12 07:08:26 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518506, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:26.117429 7221 init.cc:377] Logged in from keytab as oryx/127.31.156.66@KRBTEST.COM (short username oryx)
I20260812 07:08:26.118810 7221 webserver.cc:460] Webserver started at http://127.31.156.66:37829/ using document root <none> and password file <none>
I20260812 07:08:26.119537 7221 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:26.119654 7221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:26.119941 7221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:26.122520 7221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "db6816cdaa914c2f8f70a15558a04a0e"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "0133a332f31aa21d046fe0f219832954"
server_key_iv: "54a6d3097a66f1a370f827f01009bc99"
server_key_version: "encryptionkey@0"
I20260812 07:08:26.123343 7221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "db6816cdaa914c2f8f70a15558a04a0e"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "0133a332f31aa21d046fe0f219832954"
server_key_iv: "54a6d3097a66f1a370f827f01009bc99"
server_key_version: "encryptionkey@0"
I20260812 07:08:26.128098 7221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 07:08:26.131134 7238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:26.132544 7221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:26.132714 7221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "db6816cdaa914c2f8f70a15558a04a0e"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "0133a332f31aa21d046fe0f219832954"
server_key_iv: "54a6d3097a66f1a370f827f01009bc99"
server_key_version: "encryptionkey@0"
I20260812 07:08:26.132865 7221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:26.169730 7221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:26.174023 7221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:26.174304 7221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:26.175064 7221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:26.176115 7221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:26.176215 7221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:26.176304 7221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:26.176360 7221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:26.188428 7221 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:41119
I20260812 07:08:26.188477 7382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:41119 every 8 connection(s)
I20260812 07:08:26.189479 7221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:26.189821 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 7221
I20260812 07:08:26.189955 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:08:26.190295 32369 external_mini_cluster.cc:1428] Setting key 2b198918d93088372e45cad833a9037e
Aug 12 07:08:26 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518506, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.31.156.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.67@KRBTEST.COM" created.
I20260812 07:08:26.208223 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.191631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:32905 (local address 127.31.156.126:41665)
0812 07:08:26.191825 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:26.191830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:26.193096 (+ 1266us) server_negotiation.cc:407] Connection header received
0812 07:08:26.195345 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:26.195350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:26.195429 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:26.195541 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:26.197360 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.198306 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.199751 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.200197 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.200350 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.203398 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:26.203402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:26.203405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:26.203437 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:26.205148 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.205951 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.205954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.205956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.206045 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.206517 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.206521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.206523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.206866 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:26.206985 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:26.207934 (+ 949us) server_negotiation.cc:299] Negotiation successful
0812 07:08:26.208089 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:08:26.209509 7385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.192152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.66:32905)
0812 07:08:26.192887 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:26.192964 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:26.195072 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:26.195825 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:26.195837 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:26.196493 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:26.197170 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.197188 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.198546 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.198551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.199591 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.199603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.200389 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.200392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.200551 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.201328 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:26.201347 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:26.203162 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:26.205305 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.205311 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.205328 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.205822 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.206140 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.206142 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.206143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.206395 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.207010 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:26.207016 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:26.207530 (+ 514us) client_negotiation.cc:769] Sending connection context
0812 07:08:26.207806 (+ 276us) client_negotiation.cc:240] Negotiation successful
0812 07:08:26.208276 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":160,"threads_started":1}
I20260812 07:08:26.211269 7383 heartbeater.cc:347] Connected to a master server at 127.31.156.126:41665
I20260812 07:08:26.211580 7383 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:26.212183 7383 heartbeater.cc:511] Master 127.31.156.126:41665 requested a full tablet report, sending...
I20260812 07:08:26.213565 6992 ts_manager.cc:194] Registered new tserver with Master: db6816cdaa914c2f8f70a15558a04a0e (127.31.156.66:41119)
I20260812 07:08:26.214560 6992 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:32905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab.
Entry for principal oryx/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab.
I20260812 07:08:26.232566 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:41665
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab
--principal=oryx/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:26.379015 7389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:26.379350 7389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:26.379398 7389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:26.383905 7389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:26.383993 7389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:26.384059 7389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:26.390415 7389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab
--principal=oryx/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:41665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:26.391845 7389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:26.393225 7389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:26.404778 7394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:26.404874 7389 server_base.cc:1034] running on GCE node
W20260812 07:08:26.404857 7395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:26.404829 7397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:26.405673 7389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:26.406538 7389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:26.407778 7389 hybrid_clock.cc:648] HybridClock initialized: now 1786518506407732 us; error 56 us; skew 500 ppm
Aug 12 07:08:26 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518506, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:26.411442 7389 init.cc:377] Logged in from keytab as oryx/127.31.156.67@KRBTEST.COM (short username oryx)
I20260812 07:08:26.412595 7389 webserver.cc:460] Webserver started at http://127.31.156.67:36943/ using document root <none> and password file <none>
I20260812 07:08:26.413236 7389 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:26.413345 7389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:26.413605 7389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:26.415962 7389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "e2a1d23f5fed443ab9f448a0958ebbaa"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "4e2c3238a2807c037a52c39a6277064f"
server_key_iv: "e328e56335187a43c087e2f0fbbcde26"
server_key_version: "encryptionkey@0"
I20260812 07:08:26.416617 7389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "e2a1d23f5fed443ab9f448a0958ebbaa"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "4e2c3238a2807c037a52c39a6277064f"
server_key_iv: "e328e56335187a43c087e2f0fbbcde26"
server_key_version: "encryptionkey@0"
I20260812 07:08:26.421039 7389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:26.424259 7405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:26.425946 7389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:26.426149 7389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "e2a1d23f5fed443ab9f448a0958ebbaa"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "4e2c3238a2807c037a52c39a6277064f"
server_key_iv: "e328e56335187a43c087e2f0fbbcde26"
server_key_version: "encryptionkey@0"
I20260812 07:08:26.426306 7389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:26.465015 7389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:26.469045 7389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:26.469241 7389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:26.470032 7389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:26.471032 7389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:26.471087 7389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:26.471167 7389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:26.471206 7389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:26.482043 7389 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:42147
I20260812 07:08:26.482056 7540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:42147 every 8 connection(s)
I20260812 07:08:26.483173 7389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:26.483649 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 7389
I20260812 07:08:26.483791 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:08:26.484159 32369 external_mini_cluster.cc:1428] Setting key 6406181288aa56295078e9b0485d2c65
Aug 12 07:08:26 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518506, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:26.501044 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.485887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:47585 (local address 127.31.156.126:41665)
0812 07:08:26.486035 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:26.486040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:26.487468 (+ 1428us) server_negotiation.cc:407] Connection header received
0812 07:08:26.489065 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:26.489068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:26.489123 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:26.489209 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:26.490599 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.491642 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.493345 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.493696 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.493822 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.496928 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:26.496931 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:26.496933 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:26.496963 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:26.498771 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.499372 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.499376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.499378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.499466 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.499838 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.499843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.499848 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.500052 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:26.500167 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:26.500741 (+ 574us) server_negotiation.cc:299] Negotiation successful
0812 07:08:26.500877 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:08:26.502010 7543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.486473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.67:47585)
0812 07:08:26.487276 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:26.487346 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:26.488772 (+ 1426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:26.489388 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:26.489396 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:26.489882 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:26.490436 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.490451 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.491822 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.491826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.493193 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.493205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.493884 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.493888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.494081 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.494889 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:26.494908 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:26.496709 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:26.498903 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.498910 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.498935 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.499242 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.499567 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.499570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.499572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.499732 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.500166 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:26.500176 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:26.500459 (+ 283us) client_negotiation.cc:769] Sending connection context
0812 07:08:26.500667 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 07:08:26.501044 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":307,"threads_started":1}
I20260812 07:08:26.503293 7541 heartbeater.cc:347] Connected to a master server at 127.31.156.126:41665
I20260812 07:08:26.503584 7541 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:26.504149 7541 heartbeater.cc:511] Master 127.31.156.126:41665 requested a full tablet report, sending...
I20260812 07:08:26.505370 6992 ts_manager.cc:194] Registered new tserver with Master: e2a1d23f5fed443ab9f448a0958ebbaa (127.31.156.67:42147)
I20260812 07:08:26.505983 6992 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:47585
I20260812 07:08:26.511466 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.156.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.125.keytab.
Entry for principal oryx/127.31.156.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.125.keytab.
I20260812 07:08:26.548379 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.125:46853
--webserver_interface=127.31.156.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.125.keytab
--principal=oryx/127.31.156.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:41665,127.31.156.125:46853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:26.677858 7546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:26.678546 7546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:26.678763 7546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:26.686633 7546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:26.686872 7546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:26.686986 7546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:26.687083 7546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:26.687156 7546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:26.698846 7546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:41665,127.31.156.125:46853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.125.keytab
--principal=oryx/127.31.156.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.125:46853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.156.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
W20260812 07:08:26.699400 7546 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.156.126:41665,127.31.156.125:46853'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 07:08:26.701081 7546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:26.703032 7546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:26.717528 7555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:26.717528 7552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:26.717612 7546 server_base.cc:1034] running on GCE node
W20260812 07:08:26.717528 7553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:26.719250 7546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:26.721494 7546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:26.723073 7546 hybrid_clock.cc:648] HybridClock initialized: now 1786518506723022 us; error 60 us; skew 500 ppm
Aug 12 07:08:26 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518506, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:26.728636 7546 init.cc:377] Logged in from keytab as oryx/127.31.156.125@KRBTEST.COM (short username oryx)
I20260812 07:08:26.730594 7546 webserver.cc:460] Webserver started at http://127.31.156.125:34039/ using document root <none> and password file <none>
I20260812 07:08:26.731660 7546 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:26.731784 7546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:26.732241 7546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:26.736282 7546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data/instance:
uuid: "6f7dfd08c77349e598acda2b6757f016"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "9ab215c0123a529b51e1c736c13f59cb"
server_key_iv: "74baaa23605bf66f53690f429b77781a"
server_key_version: "encryptionkey@0"
I20260812 07:08:26.737624 7546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/wal/instance:
uuid: "6f7dfd08c77349e598acda2b6757f016"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "9ab215c0123a529b51e1c736c13f59cb"
server_key_iv: "74baaa23605bf66f53690f429b77781a"
server_key_version: "encryptionkey@0"
I20260812 07:08:26.745247 7546 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260812 07:08:26.750602 7562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:26.753772 7546 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260812 07:08:26.754067 7546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/wal
uuid: "6f7dfd08c77349e598acda2b6757f016"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "9ab215c0123a529b51e1c736c13f59cb"
server_key_iv: "74baaa23605bf66f53690f429b77781a"
server_key_version: "encryptionkey@0"
I20260812 07:08:26.754346 7546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:26.791373 7546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:26.797410 7546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:26.797765 7546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:26.810672 7546 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.125:46853
I20260812 07:08:26.811229 7620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.125:46853 every 8 connection(s)
I20260812 07:08:26.812971 7546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
I20260812 07:08:26.818984 7621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:26.820071 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 7546
I20260812 07:08:26.828866 7621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:26 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.156.125@KRBTEST.COM
Aug 12 07:08:26 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518506, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.125@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:26.851132 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.823583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58504 (local address 127.31.156.125:46853)
0812 07:08:26.824596 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:26.824647 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:26.824732 (+ 85us) server_negotiation.cc:407] Connection header received
0812 07:08:26.825984 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:26.826030 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:26.826853 (+ 823us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:26.827668 (+ 815us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:26.829312 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.831425 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.833468 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.834721 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.835160 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.841193 (+ 6033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:26.841214 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:26.841230 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:26.841281 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:26.845262 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.846523 (+ 1261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.846531 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.846547 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.846917 (+ 370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.847482 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.847487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.847489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.848200 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:26.848581 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:26.849147 (+ 566us) server_negotiation.cc:299] Negotiation successful
0812 07:08:26.849722 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":137,"server-negotiator.queue_time_us":709,"thread_start_us":295,"threads_started":1}
I20260812 07:08:26.852077 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.832324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59716 (local address 127.31.156.126:41665)
0812 07:08:26.832618 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:26.832625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:26.833374 (+ 749us) server_negotiation.cc:407] Connection header received
0812 07:08:26.833773 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:26.833777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:26.833869 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:26.834041 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:26.835825 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.836922 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.839376 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.840017 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.840203 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.845901 (+ 5698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:26.845919 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:26.845932 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:26.846019 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:26.848915 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.849745 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.849748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.849750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.849885 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.850366 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.850378 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.850380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.850778 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:26.850963 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:26.851775 (+ 812us) server_negotiation.cc:299] Negotiation successful
0812 07:08:26.851935 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 07:08:26.852104 7625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.832276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:59716)
0812 07:08:26.833116 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:26.833223 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:26.833551 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:26.834088 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:26.834104 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:26.835038 (+ 934us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:26.835548 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.835566 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.837136 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.837140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.839192 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.839215 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.840456 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.840464 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.840770 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.841799 (+ 1029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:26.841849 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:26.845507 (+ 3658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:26.849136 (+ 3629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.849158 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.849178 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.849553 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.850077 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.850080 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.850091 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.850201 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.850958 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:26.850977 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:26.851166 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 07:08:26.851442 (+ 276us) client_negotiation.cc:240] Negotiation successful
0812 07:08:26.851798 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":87,"thread_start_us":326,"threads_started":1}
I20260812 07:08:26.854322 7621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:26 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518506, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.125@KRBTEST.COM for oryx/127.31.156.125@KRBTEST.COM
I20260812 07:08:26.869647 7625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.855256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46853 (local address 127.0.0.1:58508)
0812 07:08:26.855460 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:26.855478 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:26.855616 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:26.856005 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:26.856019 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:26.856317 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:26.856454 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.856461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.858226 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.858232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.859080 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.859091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.859862 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.859868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.860021 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.860825 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:26.860837 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:26.863683 (+ 2846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:26.867268 (+ 3585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.867272 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.867274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.867637 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.868162 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.868165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.868166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.868316 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.868971 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:26.868973 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:26.869105 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 07:08:26.869230 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 07:08:26.869417 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 07:08:26.869812 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.855287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58508 (local address 127.31.156.125:46853)
0812 07:08:26.855461 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:26.855467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:26.855544 (+ 77us) server_negotiation.cc:407] Connection header received
0812 07:08:26.855809 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:26.855812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:26.855869 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:26.855997 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:26.856640 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.857983 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.859295 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.859635 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.859850 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.864006 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:26.864020 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:26.864023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:26.864060 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:26.867054 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.867829 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.867842 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.867844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.867996 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.868480 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.868483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.868484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.868791 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:26.868955 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:26.869353 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 07:08:26.869537 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:08:26.875804 7621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: Bootstrap starting.
I20260812 07:08:26.880494 7621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:26.882071 7621 log.cc:824] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:26.886247 7621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: No bootstrap required, opened a new log
I20260812 07:08:26.892827 7621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } }
I20260812 07:08:26.893378 7621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:26.893527 7621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7dfd08c77349e598acda2b6757f016, State: Initialized, Role: FOLLOWER
I20260812 07:08:26.894325 7621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } }
I20260812 07:08:26.895304 7627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } } }
I20260812 07:08:26.895603 7627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:26.896891 7621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:08:26.905993 7643 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:08:26.906174 7643 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:08:26.911051 7546 master_runner.cc:428] Detected that this master 6f7dfd08c77349e598acda2b6757f016 is joining an existing cluster
I20260812 07:08:26.911201 7546 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.156.125:46853
WARNING: no policy specified for oryx/127.31.156.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.124@KRBTEST.COM" created.
I20260812 07:08:26.933808 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.915950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:59722)
0812 07:08:26.917636 (+ 1686us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:26.917672 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:26.917932 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:26.918601 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:26.918604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:26.919094 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:26.919368 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.919379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.920896 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.920901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.922388 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.922409 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.923765 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.923779 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.923930 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.924862 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:26.924881 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:26.925843 (+ 962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:26.930311 (+ 4468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.930326 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.930328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.930951 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.931547 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.931550 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.931551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.931708 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.932568 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:26.932583 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:26.932767 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 07:08:26.933004 (+ 237us) client_negotiation.cc:240] Negotiation successful
0812 07:08:26.933417 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1552,"mutex_wait_us":4,"thread_start_us":216,"threads_started":1}
I20260812 07:08:26.934922 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.916055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.31.156.126:41665)
0812 07:08:26.916327 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:26.916334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:26.917784 (+ 1450us) server_negotiation.cc:407] Connection header received
0812 07:08:26.918127 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:26.918142 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:26.918263 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:26.918356 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:26.919527 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.920641 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.922712 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.923249 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.923436 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.926149 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:26.926155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:26.926169 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:26.926205 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:26.930053 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.931075 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.931082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.931084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.931226 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.931879 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.931883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.931885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.932368 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:26.932549 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:26.934366 (+ 1817us) server_negotiation.cc:299] Negotiation successful
0812 07:08:26.934627 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260812 07:08:26.942019 7214 heartbeater.cc:503] Master 127.31.156.126:41665 was elected leader, sending a full tablet report...
I20260812 07:08:26.945986 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.917013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58512 (local address 127.31.156.125:46853)
0812 07:08:26.917195 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:26.917203 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:26.928180 (+ 10977us) server_negotiation.cc:407] Connection header received
0812 07:08:26.928548 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:26.928551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:26.928625 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:26.928721 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:26.930990 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.932121 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.933954 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.934530 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.934761 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.938754 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:26.938761 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:26.938763 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:26.938812 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:26.942981 (+ 4169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.944054 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.944058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.944059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.944212 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.944822 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.944825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.944827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.945245 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:26.945369 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:26.945658 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 07:08:26.945799 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:08:26.946689 7652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.916998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46853 (local address 127.0.0.1:58512)
0812 07:08:26.927241 (+ 10243us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:26.928061 (+ 820us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:26.928356 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:26.929790 (+ 1434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:26.929793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:26.930325 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:26.930702 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.930722 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.932308 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.932311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.933693 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.933719 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.935079 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.935087 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.935318 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.936543 (+ 1225us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:26.936596 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:26.938484 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:26.943223 (+ 4739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.943230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.943247 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.943829 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.944423 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.944428 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.944433 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.944605 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.945419 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:26.945422 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:26.945509 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 07:08:26.946184 (+ 675us) client_negotiation.cc:240] Negotiation successful
0812 07:08:26.946386 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7533,"thread_start_us":214,"threads_started":1}
Aug 12 07:08:26 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518505, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.126@KRBTEST.COM for oryx/127.31.156.125@KRBTEST.COM
I20260812 07:08:26.960405 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.947270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58514 (local address 127.31.156.125:46853)
0812 07:08:26.947378 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:26.947383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:26.947771 (+ 388us) server_negotiation.cc:407] Connection header received
0812 07:08:26.948030 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:26.948034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:26.948088 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:26.948150 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:26.949152 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.949987 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.951567 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.952025 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.952133 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.956048 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:26.956051 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:26.956053 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:26.956081 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:26.957965 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.958615 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.958619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.958621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.958780 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:26.959154 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:26.959157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:26.959159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:26.959384 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:26.959504 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:26.959961 (+ 457us) server_negotiation.cc:299] Negotiation successful
0812 07:08:26.960104 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 07:08:26.960494 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:26.946412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46853 (local address 127.0.0.1:58514)
0812 07:08:26.947633 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:26.947697 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:26.947878 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:26.948335 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:26.948341 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:26.948843 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:26.948990 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.948998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.950148 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.950152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.951378 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:26.951402 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:26.952186 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:26.952190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:26.952330 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:26.953130 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:26.953148 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:26.955842 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:26.958149 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.958164 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.958173 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.958483 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.958893 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:26.958920 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:26.958922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:26.959036 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:26.959509 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:26.959520 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:26.959719 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 07:08:26.959889 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 07:08:26.960190 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"thread_start_us":112,"threads_started":1}
I20260812 07:08:26.962512 6992 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.156.125:46853: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6f7dfd08c77349e598acda2b6757f016"
member_type: NON_VOTER
last_known_addr {
host: "127.31.156.125"
port: 46853
}
attrs {
promote: true
}
}
dest_uuid: "0b9ef89962f743399485ce3efb4a26d8"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.124.keytab.
Entry for principal oryx/127.31.156.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.124.keytab.
I20260812 07:08:26.963490 6992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } }
I20260812 07:08:26.964115 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.124:45641
--webserver_interface=127.31.156.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.124.keytab
--principal=oryx/127.31.156.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:41665,127.31.156.125:46853,127.31.156.124:45641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:08:26.966015 7658 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6f7dfd08c77349e598acda2b6757f016 (127.31.156.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } }
I20260812 07:08:26.966876 7593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:26.968214 7658 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.156.125:46853
I20260812 07:08:26.968534 7593 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Refusing update from remote peer 0b9ef89962f743399485ce3efb4a26d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 07:08:26.968524 7658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } }
I20260812 07:08:26.968636 7658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:26.968845 7546 master.cc:537] Master@127.31.156.125:46853 shutting down...
I20260812 07:08:26.970603 7658 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:26.973661 7546 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:26.974054 7546 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:26.974188 7546 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: stopping tablet replica
W20260812 07:08:26.975615 6973 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 07:08:26.977182 6973 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 -> Peer 6f7dfd08c77349e598acda2b6757f016 (127.31.156.125:46853): Couldn't send request to peer 6f7dfd08c77349e598acda2b6757f016. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 07:08:26.991420 7546 master.cc:552] Master@127.31.156.125:46853 shutdown complete.
I20260812 07:08:26.991495 7546 master_runner.cc:305] Clearing existing system tablet
I20260812 07:08:26.991777 7546 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:26.991838 7546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:26.993440 7546 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 07:08:26.994421 7665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:26.994647 7546 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:26.994860 7546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 07:08:26.994897 7546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/wal
uuid: "6f7dfd08c77349e598acda2b6757f016"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "9ab215c0123a529b51e1c736c13f59cb"
server_key_iv: "74baaa23605bf66f53690f429b77781a"
server_key_version: "encryptionkey@0"
I20260812 07:08:26.997184 7546 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:27.000089 7546 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 07:08:27.000208 7546 log.cc:1196] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 07:08:27.000484 7546 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: Deleting consensus metadata
I20260812 07:08:27.000990 7546 master_runner.cc:315] Copying system tablet from 127.31.156.126:41665
I20260812 07:08:27.002269 7546 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: tablet copy: Beginning tablet copy session from remote peer at address 127.31.156.126:41665
I20260812 07:08:27.012616 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.003100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:59728)
0812 07:08:27.003464 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.003480 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.003607 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.003912 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.003916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.004149 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.004315 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.004324 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.005532 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.005536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.006452 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.006464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.007143 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.007147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.007252 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.007902 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.007918 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.008396 (+ 478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.010635 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.010639 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.010641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.011005 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.011345 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.011349 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.011351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.011455 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.011973 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.011977 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.012103 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.012219 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.012397 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":115,"threads_started":1}
I20260812 07:08:27.012602 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.003241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59728 (local address 127.31.156.126:41665)
0812 07:08:27.003384 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.003389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.003577 (+ 188us) server_negotiation.cc:407] Connection header received
0812 07:08:27.003697 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.003701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.003758 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.003866 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.004506 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.005368 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.006611 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.006998 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.007118 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.008564 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.008568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.008570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.008601 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.010491 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.011125 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.011129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.011131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.011232 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.011556 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.011559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.011561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.011869 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.012031 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.012291 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.012428 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:08:27.013573 7014 tablet_copy_service.cc:140] P 0b9ef89962f743399485ce3efb4a26d8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6f7dfd08c77349e598acda2b6757f016 ({username='oryx', principal='oryx/127.31.156.125@KRBTEST.COM'} at 127.0.0.1:59728)
I20260812 07:08:27.013775 7014 tablet_copy_service.cc:161] P 0b9ef89962f743399485ce3efb4a26d8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6f7dfd08c77349e598acda2b6757f016 at {username='oryx', principal='oryx/127.31.156.125@KRBTEST.COM'} at 127.0.0.1:59728: session id = 6f7dfd08c77349e598acda2b6757f016-00000000000000000000000000000000
I20260812 07:08:27.015704 7014 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 07:08:27.017558 7546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 07:08:27.020370 7546 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: tablet copy: Starting download of 0 data blocks...
I20260812 07:08:27.020602 7546 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: tablet copy: Starting download of 1 WAL segments...
I20260812 07:08:27.022814 7546 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 07:08:27.025142 7014 tablet_copy_service.cc:342] P 0b9ef89962f743399485ce3efb4a26d8: Request end of tablet copy session 6f7dfd08c77349e598acda2b6757f016-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.156.125@KRBTEST.COM'} at 127.0.0.1:59728
I20260812 07:08:27.025312 7014 tablet_copy_service.cc:433] P 0b9ef89962f743399485ce3efb4a26d8: ending tablet copy session 6f7dfd08c77349e598acda2b6757f016-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6f7dfd08c77349e598acda2b6757f016
I20260812 07:08:27.033980 7546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:27.036296 7673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:27.036335 7676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:27.036480 7674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:27.036844 7546 server_base.cc:1034] running on GCE node
I20260812 07:08:27.037014 7546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:27.037370 7546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:27.038542 7546 hybrid_clock.cc:648] HybridClock initialized: now 1786518507038514 us; error 48 us; skew 500 ppm
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:27.040516 7546 init.cc:377] Logged in from keytab as oryx/127.31.156.125@KRBTEST.COM (short username oryx)
I20260812 07:08:27.041087 7546 webserver.cc:460] Webserver started at http://127.31.156.125:36151/ using document root <none> and password file <none>
I20260812 07:08:27.041330 7546 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:27.041383 7546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:27.043078 7546 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 07:08:27.044281 7685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:27.044551 7546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 07:08:27.044652 7546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/wal
uuid: "6f7dfd08c77349e598acda2b6757f016"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "9ab215c0123a529b51e1c736c13f59cb"
server_key_iv: "74baaa23605bf66f53690f429b77781a"
server_key_version: "encryptionkey@0"
I20260812 07:08:27.044741 7546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:27.052812 7546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:27.055902 7546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:27.061424 7546 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.125:46853
I20260812 07:08:27.061893 7751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.125:46853 every 8 connection(s)
I20260812 07:08:27.062109 7546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
I20260812 07:08:27.064256 7752 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:08:27.065708 7752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: Bootstrap starting.
I20260812 07:08:27.080523 7752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:08:27.081151 7752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: Bootstrap complete.
I20260812 07:08:27.081926 7752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } }
I20260812 07:08:27.082103 7752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f7dfd08c77349e598acda2b6757f016, State: Initialized, Role: LEARNER
I20260812 07:08:27.082290 7752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } }
I20260812 07:08:27.082813 7755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } }
I20260812 07:08:27.082937 7752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:27.083091 7755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: This master's current role is: LEARNER
I20260812 07:08:27.087628 7767 catalog_manager.cc:1259] Loaded cluster ID: f8f3870a34c84e4791cfae7eac9fbe99
I20260812 07:08:27.087710 7767 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: loading cluster ID for follower catalog manager: success
I20260812 07:08:27.090368 7767 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: acquiring CA information for follower catalog manager: success
I20260812 07:08:27.092221 7767 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 07:08:27.122967 7660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:27.123338 7660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:27.123442 7660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:27.127851 7660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:27.127949 7660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:27.127992 7660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:27.128031 7660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:27.128070 7660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:27.134186 7660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:41665,127.31.156.125:46853,127.31.156.124:45641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.124.keytab
--principal=oryx/127.31.156.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.124:45641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.156.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:27.135519 7660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:27.136703 7660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:27.147213 7772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:27.147341 7773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:27.147212 7776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:27.147591 7660 server_base.cc:1034] running on GCE node
I20260812 07:08:27.148492 7660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:27.149797 7660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:27.151010 7660 hybrid_clock.cc:648] HybridClock initialized: now 1786518507150994 us; error 33 us; skew 500 ppm
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:27.154170 7660 init.cc:377] Logged in from keytab as oryx/127.31.156.124@KRBTEST.COM (short username oryx)
I20260812 07:08:27.155264 7660 webserver.cc:460] Webserver started at http://127.31.156.124:42375/ using document root <none> and password file <none>
I20260812 07:08:27.155889 7660 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:27.155992 7660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:27.156245 7660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:27.158589 7660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data/instance:
uuid: "5b48d97e4ffa4a9589bb449878f0e118"
format_stamp: "Formatted at 2026-08-12 07:08:27 on dist-test-slave-csg5"
server_key: "f2769f6d397bd4472dba66158d516a13"
server_key_iv: "f16c0dfb2e90416b28e3e6e1d22e6c24"
server_key_version: "encryptionkey@0"
I20260812 07:08:27.159329 7660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/wal/instance:
uuid: "5b48d97e4ffa4a9589bb449878f0e118"
format_stamp: "Formatted at 2026-08-12 07:08:27 on dist-test-slave-csg5"
server_key: "f2769f6d397bd4472dba66158d516a13"
server_key_iv: "f16c0dfb2e90416b28e3e6e1d22e6c24"
server_key_version: "encryptionkey@0"
I20260812 07:08:27.163900 7660 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:08:27.167147 7783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:27.168694 7660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:08:27.168910 7660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/wal
uuid: "5b48d97e4ffa4a9589bb449878f0e118"
format_stamp: "Formatted at 2026-08-12 07:08:27 on dist-test-slave-csg5"
server_key: "f2769f6d397bd4472dba66158d516a13"
server_key_iv: "f16c0dfb2e90416b28e3e6e1d22e6c24"
server_key_version: "encryptionkey@0"
I20260812 07:08:27.169072 7660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:27.184999 7660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:27.188740 7660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:27.188973 7660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:27.196897 7660 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.124:45641
I20260812 07:08:27.197265 7846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.124:45641 every 8 connection(s)
I20260812 07:08:27.198113 7660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
I20260812 07:08:27.201370 7847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:27.205655 7847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:08:27.208575 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 7660
I20260812 07:08:27.217504 7383 heartbeater.cc:503] Master 127.31.156.126:41665 was elected leader, sending a full tablet report...
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.124@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.156.124@KRBTEST.COM
I20260812 07:08:27.223533 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.208052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59738 (local address 127.31.156.126:41665)
0812 07:08:27.208314 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.208321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.209325 (+ 1004us) server_negotiation.cc:407] Connection header received
0812 07:08:27.210742 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.210746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.210810 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.210915 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.212035 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.212608 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.214197 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.214511 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.214601 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.219114 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.219118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.219121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.219175 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.221130 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.221735 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.221739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.221740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.221841 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.222228 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.222231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.222233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.222449 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.222565 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.223192 (+ 627us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.223332 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260812 07:08:27.224476 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.208440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:59738)
0812 07:08:27.209119 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.209183 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.210395 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.211000 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.211009 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.211496 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.211870 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.211887 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.212775 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.212818 (+ 43us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.213997 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.214008 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.214912 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.214915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.215214 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.216800 (+ 1586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.216821 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.218857 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.221264 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.221277 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.221289 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.221598 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.221947 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.221950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.221952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.222118 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.222566 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.222574 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.222936 (+ 362us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.223123 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.223482 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":86,"thread_start_us":282,"threads_started":1}
I20260812 07:08:27.224483 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.209638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55302 (local address 127.31.156.124:45641)
0812 07:08:27.210015 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.210024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.210062 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:08:27.210450 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.210483 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.210929 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.211121 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.212085 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.213332 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.214762 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.215482 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.215612 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.219621 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.219628 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.219642 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.219667 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.221486 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.222011 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.222017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.222018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.222121 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.222456 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.222460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.222462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.222957 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.223212 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.223386 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.223542 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":263,"spinlock_wait_cycles":1280,"thread_start_us":138,"threads_started":1}
I20260812 07:08:27.226257 7847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.124@KRBTEST.COM for oryx/127.31.156.125@KRBTEST.COM
I20260812 07:08:27.237329 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.226923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46853 (local address 127.0.0.1:58528)
0812 07:08:27.227039 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.227055 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.227142 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.227586 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.227588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.227823 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.227938 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.227944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.228986 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.228989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.229601 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.229610 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.230350 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.230353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.230457 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.231030 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.231042 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.232748 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.235233 (+ 2485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.235237 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.235239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.235528 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.235859 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.235862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.235864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.236010 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.236565 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.236568 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.236690 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.236809 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.236969 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 07:08:27.237349 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.227055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58528 (local address 127.31.156.125:46853)
0812 07:08:27.227341 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.227345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.227357 (+ 12us) server_negotiation.cc:407] Connection header received
0812 07:08:27.227395 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.227397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.227437 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.227516 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.228094 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.228847 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.229737 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.229995 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.230191 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.232952 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.232955 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.232957 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.232997 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.235062 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.235645 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.235648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.235650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.235751 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.236146 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.236149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.236151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.236452 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.236547 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.236984 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.237146 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":106,"threads_started":1}
I20260812 07:08:27.238754 7847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.124@KRBTEST.COM for oryx/127.31.156.124@KRBTEST.COM
I20260812 07:08:27.249547 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.239334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:45641 (local address 127.0.0.1:55314)
0812 07:08:27.239475 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.239493 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.239580 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.239926 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.239929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.240103 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.240192 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.240197 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.241212 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.241215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.241887 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.241898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.242560 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.242563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.242631 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.243219 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.243232 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.244746 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.247086 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.247089 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.247091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.247340 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.248046 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.248049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.248051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.248157 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.248915 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.248918 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.249092 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.249239 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.249400 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 07:08:27.249686 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.239539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.31.156.124:45641)
0812 07:08:27.239704 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.239708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.239722 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:08:27.239758 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.239760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.239797 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.239878 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.240500 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.241091 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.242013 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.242413 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.242509 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.244914 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.244918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.244920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.244942 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.246925 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.247540 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.247543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.247545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.247895 (+ 350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.248541 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.248544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.248545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.248785 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.248897 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.249393 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.249522 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:08:27.253588 7847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: Bootstrap starting.
I20260812 07:08:27.256448 7847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:27.257328 7847 log.cc:824] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:27.259900 7847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: No bootstrap required, opened a new log
I20260812 07:08:27.263656 7847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } }
I20260812 07:08:27.264000 7847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:27.264071 7847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b48d97e4ffa4a9589bb449878f0e118, State: Initialized, Role: FOLLOWER
I20260812 07:08:27.264623 7847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } }
I20260812 07:08:27.265275 7854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } } }
I20260812 07:08:27.265404 7854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:27.266140 7847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:08:27.272162 7867 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:08:27.272289 7867 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:08:27.276754 7660 master_runner.cc:428] Detected that this master 5b48d97e4ffa4a9589bb449878f0e118 is joining an existing cluster
I20260812 07:08:27.276842 7660 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.156.124:45641
I20260812 07:08:27.278326 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 7063
I20260812 07:08:27.297386 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:38327
--local_ip_for_outbound_sockets=127.31.156.65
--tserver_master_addrs=127.31.156.126:41665
--webserver_port=37231
--webserver_interface=127.31.156.65
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab
--principal=oryx/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.156.126:41665,127.31.156.125:46853
--tserver_master_addrs=127.31.156.126:41665,127.31.156.125:46853,127.31.156.124:45641 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:08:27.301101 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.281507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.31.156.126:41665)
0812 07:08:27.288510 (+ 7003us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.288515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.289845 (+ 1330us) server_negotiation.cc:407] Connection header received
0812 07:08:27.290001 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.290007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.290082 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.290210 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.291169 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.292367 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.293986 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.294452 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.294598 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.296796 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.296803 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.296809 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.296849 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.298592 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.299207 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.299211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.299218 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.299363 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.299753 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.299758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.299762 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.300046 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.300143 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.300684 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.300867 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6888}
I20260812 07:08:27.301796 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.280907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:59742)
0812 07:08:27.289688 (+ 8781us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.289709 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.289890 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.290280 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.290284 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.290660 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.290984 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.290998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.292555 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.292563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.293801 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.293822 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.294644 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.294648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.294806 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.295962 (+ 1156us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.295984 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.296532 (+ 548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.298816 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.298820 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.298822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.299078 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.299508 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.299513 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.299517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.299637 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.300218 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.300222 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.300381 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.301194 (+ 813us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.301576 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8654,"thread_start_us":158,"threads_started":1}
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518505, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.126@KRBTEST.COM for oryx/127.31.156.124@KRBTEST.COM
I20260812 07:08:27.323276 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.306286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:45641 (local address 127.0.0.1:55332)
0812 07:08:27.306496 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.306512 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.306603 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.307175 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.307179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.307403 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.307534 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.307541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.308454 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.308459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.309168 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.309179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.309706 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.309710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.309811 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.310348 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.310369 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.311874 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.321252 (+ 9378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.321257 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.321259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.321584 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.321961 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.321964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.321967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.322060 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.322627 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.322631 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.322787 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.322878 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.323057 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260812 07:08:27.326319 7879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.306343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55332 (local address 127.31.156.124:45641)
0812 07:08:27.306876 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.306885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.306899 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:08:27.306973 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.306975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.307028 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.307656 (+ 628us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.307780 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.308332 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.309298 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.309571 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.309686 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.312057 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.312062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.312065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.312094 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.321072 (+ 8978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.321742 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.321745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.321748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.321845 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.322171 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.322176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.322180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.322506 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.323970 (+ 1464us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.324057 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.326057 (+ 2000us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":119,"threads_started":1}
I20260812 07:08:27.327603 6992 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.156.124:45641: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118"
member_type: NON_VOTER
last_known_addr {
host: "127.31.156.124"
port: 45641
}
attrs {
promote: true
}
}
dest_uuid: "0b9ef89962f743399485ce3efb4a26d8"
cas_config_opid_index: 5
I20260812 07:08:27.328405 7875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.288187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:45641 (local address 127.0.0.1:55330)
0812 07:08:27.295370 (+ 7183us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.295387 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.295531 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.295914 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.295919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.319100 (+ 23181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.319367 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.319382 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.320643 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.320650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.321842 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.321859 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.322758 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.322762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.322868 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.323467 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.323484 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.324006 (+ 522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.326341 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.326343 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.326346 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.326907 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.327184 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.327186 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.327188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.327276 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.327686 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.327689 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.327765 (+ 76us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.328037 (+ 272us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.328189 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7094,"mutex_wait_us":4,"spinlock_wait_cycles":16000,"thread_start_us":84,"threads_started":1}
I20260812 07:08:27.328414 6992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } }
I20260812 07:08:27.329118 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.288283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.31.156.124:45641)
0812 07:08:27.290959 (+ 2676us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.290967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.295462 (+ 4495us) server_negotiation.cc:407] Connection header received
0812 07:08:27.295627 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.295630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.295669 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.295746 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.319573 (+ 23827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.320483 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.322009 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.322461 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.322570 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.324162 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.324166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.324169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.324191 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.326212 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.327018 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.327021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.327023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.327098 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.327360 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.327363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.327364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.327569 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.327654 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.328837 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.328969 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":2457}
I20260812 07:08:27.330276 7659 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5b48d97e4ffa4a9589bb449878f0e118 (127.31.156.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.333570 7817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:27.335704 7817 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 FOLLOWER]: Refusing update from remote peer 0b9ef89962f743399485ce3efb4a26d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 07:08:27.335960 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.287669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58540 (local address 127.31.156.125:46853)
0812 07:08:27.287905 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.287912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.295052 (+ 7140us) server_negotiation.cc:407] Connection header received
0812 07:08:27.295201 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.295207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.295290 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.295382 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.319626 (+ 24244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.320550 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.324607 (+ 4057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.324995 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.325078 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.330529 (+ 5451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.330534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.330537 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.330572 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.332783 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.333684 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.333690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.333692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.333874 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.334591 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.334599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.334602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.335023 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.335135 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.335555 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.335751 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 07:08:27.336510 7658 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:27.337237 7874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.287563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46853 (local address 127.0.0.1:58540)
0812 07:08:27.294907 (+ 7344us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.294929 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.295080 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.295560 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.295566 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.319103 (+ 23537us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.319393 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.319408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.320759 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.320764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.324428 (+ 3664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.324439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.325277 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.325280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.325383 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.325873 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.325888 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.330297 (+ 4409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.332946 (+ 2649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.332949 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.332952 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.333387 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.334073 (+ 686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.334075 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.334078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.334406 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.335209 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.335212 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.335334 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.336639 (+ 1305us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.336934 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7229,"mutex_wait_us":2727,"spinlock_wait_cycles":11648,"thread_start_us":173,"threads_started":1}
I20260812 07:08:27.339872 7659 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.156.124:45641
I20260812 07:08:27.340147 7659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.340273 7659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:27.343250 7660 master.cc:537] Master@127.31.156.124:45641 shutting down...
I20260812 07:08:27.346637 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.331048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46853 (local address 127.0.0.1:58556)
0812 07:08:27.340746 (+ 9698us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.340767 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.340907 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.341464 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.341468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.341541 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:27.341651 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.341658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.342910 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.342914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.344616 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.344632 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.346055 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.346060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.346198 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.346261 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.346362 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.346463 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9582}
I20260812 07:08:27.347319 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.331337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58556 (local address 127.31.156.125:46853)
0812 07:08:27.336162 (+ 4825us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.336179 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.341068 (+ 4889us) server_negotiation.cc:407] Connection header received
0812 07:08:27.341130 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.341133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.341205 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.341299 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:27.341796 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.342735 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.344805 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.345605 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.345763 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.345968 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.347063 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.347170 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4752,"thread_start_us":112,"threads_started":1}
I20260812 07:08:27.348412 7719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 LEARNER]: Refusing update from remote peer 0b9ef89962f743399485ce3efb4a26d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 07:08:27.348989 7880 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 07:08:27.349083 7816 raft_consensus.cc:858] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } }
W20260812 07:08:27.349401 7816 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } }
I20260812 07:08:27.351236 7816 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6f7dfd08c77349e598acda2b6757f016 (127.31.156.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } }
I20260812 07:08:27.351307 7719 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5b48d97e4ffa4a9589bb449878f0e118 (127.31.156.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.353681 7755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b9ef89962f743399485ce3efb4a26d8. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.353837 7755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: This master's current role is: LEARNER
I20260812 07:08:27.353936 7755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.354029 7755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: This master's current role is: LEARNER
I20260812 07:08:27.354765 7854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b9ef89962f743399485ce3efb4a26d8. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } }
I20260812 07:08:27.354914 7854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 07:08:27.355033 7854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } }
I20260812 07:08:27.355136 7854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 07:08:27.355219 7854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: NON_VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: true } } }
I20260812 07:08:27.355329 7854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 07:08:27.357146 7882 mvcc.cc:205] Tried to move back new op lower bound from 7317579799092682752 to 7317579798900953088. Current Snapshot: MvccSnapshot[applied={T|T < 7317579798943928320}]
I20260812 07:08:27.357291 7658 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8: attempting to promote NON_VOTER 6f7dfd08c77349e598acda2b6757f016 to VOTER
I20260812 07:08:27.357777 7658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } }
I20260812 07:08:27.360445 7719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 LEARNER]: Refusing update from remote peer 0b9ef89962f743399485ce3efb4a26d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 07:08:27.361635 7659 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 07:08:27.362239 7660 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260812 07:08:27.362602 7660 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
W20260812 07:08:27.362643 6973 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 -> Peer 5b48d97e4ffa4a9589bb449878f0e118 (127.31.156.124:45641): Couldn't send request to peer 5b48d97e4ffa4a9589bb449878f0e118. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 07:08:27.362766 7660 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: stopping tablet replica
I20260812 07:08:27.365123 7659 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6f7dfd08c77349e598acda2b6757f016 (127.31.156.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.367091 7659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.367261 7659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:27.367345 7880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.367437 7880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:27.367849 7719 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6f7dfd08c77349e598acda2b6757f016 (127.31.156.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.369426 7887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.369534 7887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:27.383669 7660 master.cc:552] Master@127.31.156.124:45641 shutdown complete.
I20260812 07:08:27.383814 7660 master_runner.cc:305] Clearing existing system tablet
I20260812 07:08:27.384090 7660 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:27.384174 7660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:27.385759 7660 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 07:08:27.386890 7894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:27.387163 7660 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:27.387315 7660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 07:08:27.387367 7660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/wal
uuid: "5b48d97e4ffa4a9589bb449878f0e118"
format_stamp: "Formatted at 2026-08-12 07:08:27 on dist-test-slave-csg5"
server_key: "f2769f6d397bd4472dba66158d516a13"
server_key_iv: "f16c0dfb2e90416b28e3e6e1d22e6c24"
server_key_version: "encryptionkey@0"
I20260812 07:08:27.389146 7660 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:27.391669 7660 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 07:08:27.391819 7660 log.cc:1196] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 07:08:27.392127 7660 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: Deleting consensus metadata
I20260812 07:08:27.392351 7660 master_runner.cc:315] Copying system tablet from 127.31.156.126:41665
I20260812 07:08:27.393807 7660 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: tablet copy: Beginning tablet copy session from remote peer at address 127.31.156.126:41665
I20260812 07:08:27.403304 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.394531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.31.156.126:41665)
0812 07:08:27.394741 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.394746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.394966 (+ 220us) server_negotiation.cc:407] Connection header received
0812 07:08:27.395140 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.395143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.395199 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.395345 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.395893 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.396511 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.397484 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.397758 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.397897 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.399359 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.399362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.399366 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.399404 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.401209 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.401756 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.401760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.401763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.401861 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.402204 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.402209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.402211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.402440 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.402566 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.402953 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.403094 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:08:27.403316 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.394454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:59770)
0812 07:08:27.394860 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.394874 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.395024 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.395368 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.395371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.395635 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.395740 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.395746 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.396652 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.396658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.397341 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.397351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.397907 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.397911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.398008 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.398716 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.398739 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.399177 (+ 438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.401368 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.401373 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.401375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.401631 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.401983 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.401987 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.401989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.402097 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.402569 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.402573 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.402730 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.402872 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.403075 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":113,"threads_started":1}
I20260812 07:08:27.404206 7014 tablet_copy_service.cc:140] P 0b9ef89962f743399485ce3efb4a26d8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5b48d97e4ffa4a9589bb449878f0e118 ({username='oryx', principal='oryx/127.31.156.124@KRBTEST.COM'} at 127.0.0.1:59770)
I20260812 07:08:27.404345 7014 tablet_copy_service.cc:161] P 0b9ef89962f743399485ce3efb4a26d8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5b48d97e4ffa4a9589bb449878f0e118 at {username='oryx', principal='oryx/127.31.156.124@KRBTEST.COM'} at 127.0.0.1:59770: session id = 5b48d97e4ffa4a9589bb449878f0e118-00000000000000000000000000000000
I20260812 07:08:27.405311 7014 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 07:08:27.406872 7660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 07:08:27.409085 7660 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: tablet copy: Starting download of 0 data blocks...
I20260812 07:08:27.409374 7660 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: tablet copy: Starting download of 1 WAL segments...
I20260812 07:08:27.411262 7660 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 07:08:27.413621 7014 tablet_copy_service.cc:342] P 0b9ef89962f743399485ce3efb4a26d8: Request end of tablet copy session 5b48d97e4ffa4a9589bb449878f0e118-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.156.124@KRBTEST.COM'} at 127.0.0.1:59770
I20260812 07:08:27.413763 7014 tablet_copy_service.cc:433] P 0b9ef89962f743399485ce3efb4a26d8: ending tablet copy session 5b48d97e4ffa4a9589bb449878f0e118-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5b48d97e4ffa4a9589bb449878f0e118
I20260812 07:08:27.422116 7660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:27.424984 7902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:27.425289 7903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:27.426823 7905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:27.426996 7660 server_base.cc:1034] running on GCE node
I20260812 07:08:27.427300 7660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:27.427711 7660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:27.428871 7660 hybrid_clock.cc:648] HybridClock initialized: now 1786518507428857 us; error 33 us; skew 500 ppm
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:27.430845 7660 init.cc:377] Logged in from keytab as oryx/127.31.156.124@KRBTEST.COM (short username oryx)
I20260812 07:08:27.431465 7660 webserver.cc:460] Webserver started at http://127.31.156.124:33323/ using document root <none> and password file <none>
I20260812 07:08:27.431705 7660 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:27.431766 7660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:27.433053 7660 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 07:08:27.434083 7913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:27.434337 7660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 07:08:27.434440 7660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/wal
uuid: "5b48d97e4ffa4a9589bb449878f0e118"
format_stamp: "Formatted at 2026-08-12 07:08:27 on dist-test-slave-csg5"
server_key: "f2769f6d397bd4472dba66158d516a13"
server_key_iv: "f16c0dfb2e90416b28e3e6e1d22e6c24"
server_key_version: "encryptionkey@0"
I20260812 07:08:27.434566 7660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:27.452010 7660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:27.454365 7660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:27.459956 7660 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.124:45641
I20260812 07:08:27.459981 7973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.124:45641 every 8 connection(s)
I20260812 07:08:27.460593 7660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
I20260812 07:08:27.462314 7974 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:08:27.463375 7974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: Bootstrap starting.
I20260812 07:08:27.470831 7974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:08:27.471197 7974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: Bootstrap complete.
I20260812 07:08:27.471803 7974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } }
I20260812 07:08:27.471953 7974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b48d97e4ffa4a9589bb449878f0e118, State: Initialized, Role: LEARNER
I20260812 07:08:27.472082 7974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } }
I20260812 07:08:27.472513 7978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.472712 7974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:27.472712 7978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: This master's current role is: LEARNER
I20260812 07:08:27.476264 7992 catalog_manager.cc:1259] Loaded cluster ID: f8f3870a34c84e4791cfae7eac9fbe99
I20260812 07:08:27.476416 7992 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: loading cluster ID for follower catalog manager: success
I20260812 07:08:27.478324 7992 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: acquiring CA information for follower catalog manager: success
I20260812 07:08:27.479651 7992 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 07:08:27.495538 7878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:27.495987 7878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:27.496068 7878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:27.501857 7878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:27.502004 7878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:27.502127 7878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:27.508567 7541 heartbeater.cc:503] Master 127.31.156.126:41665 was elected leader, sending a full tablet report...
I20260812 07:08:27.508670 7878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab
--principal=oryx/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:38327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=37231
--tserver_master_addrs=127.31.156.126:41665,127.31.156.125:46853,127.31.156.124:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:27.510177 7878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:27.511415 7878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:27.523928 8002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:27.524086 7878 server_base.cc:1034] running on GCE node
W20260812 07:08:27.523921 8000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:27.523921 7999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:27.524809 7878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:27.525595 7878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:27.526804 7878 hybrid_clock.cc:648] HybridClock initialized: now 1786518507526787 us; error 30 us; skew 500 ppm
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:27.529983 7878 init.cc:377] Logged in from keytab as oryx/127.31.156.65@KRBTEST.COM (short username oryx)
I20260812 07:08:27.531208 7878 webserver.cc:460] Webserver started at http://127.31.156.65:37231/ using document root <none> and password file <none>
I20260812 07:08:27.531958 7878 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:27.532073 7878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:27.536873 7878 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:08:27.539530 8010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:27.541141 7878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:27.541325 7878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "bfeb604fa3744b96a91e3cca5065e8ba"
format_stamp: "Formatted at 2026-08-12 07:08:25 on dist-test-slave-csg5"
server_key: "6b0ef42dde42be2448f6614d6e84539f"
server_key_iv: "4f78b4120d4aafe5d605891599cf26d9"
server_key_version: "encryptionkey@0"
I20260812 07:08:27.541994 7878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:27.564781 7878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:27.569707 7878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:27.569947 7878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:27.570713 7878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:27.571949 7878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:27.572023 7878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:27.572084 7878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:27.572113 7878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:27.583235 7878 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:38327
I20260812 07:08:27.583385 8143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:38327 every 8 connection(s)
I20260812 07:08:27.584481 7878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:27.589617 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 7878
I20260812 07:08:27.589854 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 7221
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.124@KRBTEST.COM
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:27.605489 8152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.588348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:45641 (local address 127.31.156.65:44255)
0812 07:08:27.588716 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.588731 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.589452 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.590279 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.590292 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.590983 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.591831 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.591854 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.593404 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.593411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.594900 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.594915 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.595675 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.595682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.595946 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.596830 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.596855 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.598947 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.601225 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.601234 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.601258 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.602253 (+ 995us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.602615 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.602620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.602624 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.602851 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.603355 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.603365 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.603753 (+ 388us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.603939 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.604324 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":43,"thread_start_us":90,"threads_started":1}
I20260812 07:08:27.606685 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.588603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:44255 (local address 127.31.156.124:45641)
0812 07:08:27.588974 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.588987 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.589000 (+ 13us) server_negotiation.cc:407] Connection header received
0812 07:08:27.589814 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.589817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.589897 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.591979 (+ 2082us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.592130 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.593133 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.595081 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.595480 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.596375 (+ 895us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.599205 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.599211 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.599217 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.599254 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.601075 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.602397 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.602403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.602407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.602504 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.602961 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.602966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.602971 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.603235 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.606169 (+ 2934us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.606260 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.606425 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":187,"threads_started":1}
I20260812 07:08:27.606738 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.586858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:41827 (local address 127.31.156.126:41665)
0812 07:08:27.587016 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.587021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.588140 (+ 1119us) server_negotiation.cc:407] Connection header received
0812 07:08:27.589705 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.589711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.589788 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.589854 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.592082 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.593453 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.595577 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.596189 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.596781 (+ 592us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.600908 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.600918 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.600926 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.600979 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.603511 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.604282 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.604290 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.604295 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.604429 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.604905 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.604912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.604917 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.605310 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.605452 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.606203 (+ 751us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.606421 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:08:27.607880 8150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.587105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.65:41827)
0812 07:08:27.587942 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.587995 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.589409 (+ 1414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.590281 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.590293 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.590984 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.591833 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.591858 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.593708 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.593716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.595361 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.595380 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.597044 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.597053 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.597250 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.598041 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.598069 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.600631 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.603696 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.603703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.603709 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.604090 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.604594 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.604601 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.604606 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.604742 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.605540 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.605547 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.605888 (+ 341us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.607217 (+ 1329us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.607490 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":58,"spinlock_wait_cycles":27008,"thread_start_us":147,"threads_started":1}
I20260812 07:08:27.608932 8145 heartbeater.cc:347] Connected to a master server at 127.31.156.126:41665
I20260812 07:08:27.609170 8145 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:27.609702 8145 heartbeater.cc:511] Master 127.31.156.126:41665 requested a full tablet report, sending...
I20260812 07:08:27.611162 6992 ts_manager.cc:194] Re-registered known tserver with Master: bfeb604fa3744b96a91e3cca5065e8ba (127.31.156.65:38327)
I20260812 07:08:27.611994 6992 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:41827
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.125@KRBTEST.COM
I20260812 07:08:27.619448 8144 heartbeater.cc:347] Connected to a master server at 127.31.156.124:45641
I20260812 07:08:27.619622 8144 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:27.619922 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:41119
--local_ip_for_outbound_sockets=127.31.156.66
--tserver_master_addrs=127.31.156.126:41665
--webserver_port=37829
--webserver_interface=127.31.156.66
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab
--principal=oryx/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.156.126:41665,127.31.156.125:46853
--tserver_master_addrs=127.31.156.126:41665,127.31.156.125:46853,127.31.156.124:45641 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:08:27.619973 8144 heartbeater.cc:511] Master 127.31.156.124:45641 requested a full tablet report, sending...
I20260812 07:08:27.621592 7931 ts_manager.cc:194] Registered new tserver with Master: bfeb604fa3744b96a91e3cca5065e8ba (127.31.156.65:38327)
I20260812 07:08:27.622083 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.587105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46853 (local address 127.31.156.65:60517)
0812 07:08:27.587931 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.587992 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.589429 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.610836 (+ 21407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.610839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.611073 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.611225 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.611249 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.612339 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.612346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.613179 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.613191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.614105 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.614109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.614204 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.614872 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.614884 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.616745 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.619671 (+ 2926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.619676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.619678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.619968 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.620364 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.620368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.620370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.620479 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.621163 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.621169 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.621323 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.621438 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.621677 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":22,"spinlock_wait_cycles":12800,"thread_start_us":161,"threads_started":1}
I20260812 07:08:27.622900 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.586582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:60517 (local address 127.31.156.125:46853)
0812 07:08:27.586783 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.586788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.588109 (+ 1321us) server_negotiation.cc:407] Connection header received
0812 07:08:27.589800 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.589804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.589868 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.589952 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.611426 (+ 21474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.612200 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.613425 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.613922 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.614073 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.616986 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.616994 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.617025 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.617071 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.619499 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.620114 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.620118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.620120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.620245 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.620627 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.620634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.620639 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.620967 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.621104 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.622488 (+ 1384us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.622679 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 07:08:27.623651 8147 heartbeater.cc:347] Connected to a master server at 127.31.156.125:46853
I20260812 07:08:27.623768 8147 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:27.623976 8147 heartbeater.cc:511] Master 127.31.156.125:46853 requested a full tablet report, sending...
I20260812 07:08:27.625339 7705 ts_manager.cc:194] Registered new tserver with Master: bfeb604fa3744b96a91e3cca5065e8ba (127.31.156.65:38327)
W20260812 07:08:27.753926 8154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:27.754338 8154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:27.754451 8154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:27.760962 8154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:27.761088 8154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:27.761214 8154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:27.767967 8154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab
--principal=oryx/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:41119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=37829
--tserver_master_addrs=127.31.156.126:41665,127.31.156.125:46853,127.31.156.124:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:27.769292 8154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:27.770741 8154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:27.782830 8165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:27.782833 8163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:27.782830 8162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:27.783495 8154 server_base.cc:1034] running on GCE node
I20260812 07:08:27.783981 8154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:27.784773 8154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:27.786005 8154 hybrid_clock.cc:648] HybridClock initialized: now 1786518507785987 us; error 39 us; skew 500 ppm
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:27.789878 8154 init.cc:377] Logged in from keytab as oryx/127.31.156.66@KRBTEST.COM (short username oryx)
I20260812 07:08:27.791085 8154 webserver.cc:460] Webserver started at http://127.31.156.66:37829/ using document root <none> and password file <none>
I20260812 07:08:27.791780 8154 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:27.791915 8154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:27.796746 8154 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 07:08:27.799561 8175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:27.801339 8154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:27.801550 8154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "db6816cdaa914c2f8f70a15558a04a0e"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "0133a332f31aa21d046fe0f219832954"
server_key_iv: "54a6d3097a66f1a370f827f01009bc99"
server_key_version: "encryptionkey@0"
I20260812 07:08:27.802346 8154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:27.819135 8154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:27.823364 8154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:27.823657 8154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:27.824405 8154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:27.825489 8154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:27.825579 8154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:27.825673 8154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:27.825735 8154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:27.836822 8154 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:41119
I20260812 07:08:27.836877 8304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:41119 every 8 connection(s)
I20260812 07:08:27.838053 8154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:27.840547 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 8154
I20260812 07:08:27.840757 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 7389
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.124@KRBTEST.COM
I20260812 07:08:27.861726 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.844875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:45641 (local address 127.0.0.1:50062)
0812 07:08:27.845056 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.845071 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.845192 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.857181 (+ 11989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.857187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.857223 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:27.857357 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.857368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.858610 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.858616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.859986 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.860005 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.861176 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.861180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.861299 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.861333 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.861446 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.861539 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.125@KRBTEST.COM
I20260812 07:08:27.862203 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.845432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.31.156.124:45641)
0812 07:08:27.856829 (+ 11397us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.856836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.856854 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:08:27.856909 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.856915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.856998 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.857494 (+ 496us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:27.857626 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.858461 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.860172 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.861018 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.861169 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.861358 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.861948 (+ 590us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.862047 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11283,"thread_start_us":379,"threads_started":1}
Aug 12 07:08:27 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518507, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:27.863060 7943 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 07:08:27.863838 7978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b9ef89962f743399485ce3efb4a26d8. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:27.864007 7978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: This master's current role is: LEARNER
I20260812 07:08:27.863971 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:42147
--local_ip_for_outbound_sockets=127.31.156.67
--tserver_master_addrs=127.31.156.126:41665
--webserver_port=36943
--webserver_interface=127.31.156.67
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab
--principal=oryx/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.156.126:41665,127.31.156.125:46853
--tserver_master_addrs=127.31.156.126:41665,127.31.156.125:46853,127.31.156.124:45641 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:08:27.868763 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.840491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:40381 (local address 127.31.156.124:45641)
0812 07:08:27.840670 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.840677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.852450 (+ 11773us) server_negotiation.cc:407] Connection header received
0812 07:08:27.852693 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.852700 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.852764 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.852824 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.853653 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.854609 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.857866 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.858257 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.858335 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.862783 (+ 4448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.862789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.862795 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.862830 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.865787 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.866893 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.866898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.866901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.867002 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.867351 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.867356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.867360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.867628 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.867719 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.868448 (+ 729us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.868595 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 07:08:27.868772 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.844029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:50171 (local address 127.31.156.125:46853)
0812 07:08:27.844244 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.844251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.854957 (+ 10706us) server_negotiation.cc:407] Connection header received
0812 07:08:27.855165 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.855169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.855229 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.855291 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.856020 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.856902 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.857999 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.858350 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.858431 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.864032 (+ 5601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.864036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.864040 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.864074 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.866052 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.866567 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.866573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.866578 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.866692 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.867152 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.867156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.867170 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.867420 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.867502 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.868446 (+ 944us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.868588 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 07:08:27.868762 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.840170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:59895 (local address 127.31.156.126:41665)
0812 07:08:27.840348 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:27.840354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:27.847109 (+ 6755us) server_negotiation.cc:407] Connection header received
0812 07:08:27.849097 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:27.849103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:27.849180 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:27.849257 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:27.854127 (+ 4870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.855007 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.857036 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.857401 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.857483 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.863352 (+ 5869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:27.863355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:27.863358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:27.863416 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:27.865634 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.866523 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.866528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.866530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.866634 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:27.867154 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:27.867158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:27.867160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:27.867427 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:27.867506 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:27.868446 (+ 940us) server_negotiation.cc:299] Negotiation successful
0812 07:08:27.868588 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:08:27.870340 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.841944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46853 (local address 127.31.156.66:50171)
0812 07:08:27.854846 (+ 12902us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.854865 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.855035 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.855453 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.855458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.855720 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.855853 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.855860 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.857028 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.857031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.857880 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.857889 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.858589 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.858592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.858847 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.859913 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.859944 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.863834 (+ 3890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.866160 (+ 2326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.866163 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.866165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.866468 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.866807 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.866810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.866811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.867031 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.867814 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.867817 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.868169 (+ 352us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.868979 (+ 810us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.869445 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12804,"mutex_wait_us":418,"spinlock_wait_cycles":28544,"thread_start_us":100,"threads_started":1}
I20260812 07:08:27.870635 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.840674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.66:59895)
0812 07:08:27.846831 (+ 6157us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.846949 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.848754 (+ 1805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.849593 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.849609 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.850514 (+ 905us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.851812 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.851853 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.855186 (+ 3333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.855192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.856896 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.856904 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.857711 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.857714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.858901 (+ 1187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.860110 (+ 1209us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.860127 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.863185 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.865821 (+ 2636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.865831 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.865856 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.866366 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.866791 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.866797 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.866799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.867031 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.867747 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.867756 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.868169 (+ 413us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.868927 (+ 758us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.869101 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5728,"mutex_wait_us":1629,"spinlock_wait_cycles":18176,"thread_start_us":167,"threads_started":1}
I20260812 07:08:27.870364 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:27.840674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:45641 (local address 127.31.156.66:40381)
0812 07:08:27.852300 (+ 11626us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:27.852326 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:27.852555 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:27.852993 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:27.852999 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:27.853308 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:27.853478 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.853490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.854857 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.854863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.856701 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:27.856712 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:27.858508 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:27.858515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:27.858849 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:27.859913 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:27.859944 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:27.862552 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:27.865947 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.865951 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.865953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.866768 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.867125 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:27.867129 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:27.867132 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:27.867243 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:27.867789 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:27.867794 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:27.868168 (+ 374us) client_negotiation.cc:769] Sending connection context
0812 07:08:27.868388 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 07:08:27.868775 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11470,"mutex_wait_us":162,"spinlock_wait_cycles":80896,"thread_start_us":144,"threads_started":1}
I20260812 07:08:27.872004 8306 heartbeater.cc:347] Connected to a master server at 127.31.156.126:41665
I20260812 07:08:27.872082 8308 heartbeater.cc:347] Connected to a master server at 127.31.156.125:46853
I20260812 07:08:27.872295 8308 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:27.872294 8306 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:27.872473 8305 heartbeater.cc:347] Connected to a master server at 127.31.156.124:45641
I20260812 07:08:27.872581 8305 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:27.873076 8308 heartbeater.cc:511] Master 127.31.156.125:46853 requested a full tablet report, sending...
I20260812 07:08:27.873095 8306 heartbeater.cc:511] Master 127.31.156.126:41665 requested a full tablet report, sending...
I20260812 07:08:27.873086 8305 heartbeater.cc:511] Master 127.31.156.124:45641 requested a full tablet report, sending...
I20260812 07:08:27.874707 6992 ts_manager.cc:194] Re-registered known tserver with Master: db6816cdaa914c2f8f70a15558a04a0e (127.31.156.66:41119)
I20260812 07:08:27.874751 7705 ts_manager.cc:194] Registered new tserver with Master: db6816cdaa914c2f8f70a15558a04a0e (127.31.156.66:41119)
I20260812 07:08:27.875348 7931 ts_manager.cc:194] Registered new tserver with Master: db6816cdaa914c2f8f70a15558a04a0e (127.31.156.66:41119)
I20260812 07:08:27.875672 6992 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:59895
W20260812 07:08:27.996692 8314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:27.997014 8314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:27.997061 8314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:28.001670 8314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:28.001777 8314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:28.001847 8314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:28.008383 8314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab
--principal=oryx/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:42147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=36943
--tserver_master_addrs=127.31.156.126:41665,127.31.156.125:46853,127.31.156.124:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:28.009792 8314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:28.011157 8314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:28.023494 8323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:28.023654 8321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:28.023702 8320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:28.023872 8314 server_base.cc:1034] running on GCE node
I20260812 07:08:28.024500 8314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:28.025445 8314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:28.026770 8314 hybrid_clock.cc:648] HybridClock initialized: now 1786518508026738 us; error 43 us; skew 500 ppm
Aug 12 07:08:28 dist-test-slave-csg5 krb5kdc[6941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518508, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:28.039692 8314 init.cc:377] Logged in from keytab as oryx/127.31.156.67@KRBTEST.COM (short username oryx)
I20260812 07:08:28.040899 8314 webserver.cc:460] Webserver started at http://127.31.156.67:36943/ using document root <none> and password file <none>
I20260812 07:08:28.041566 8314 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:28.041678 8314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:28.046478 8314 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:08:28.049104 8330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:28.050870 8314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:28.051040 8314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "e2a1d23f5fed443ab9f448a0958ebbaa"
format_stamp: "Formatted at 2026-08-12 07:08:26 on dist-test-slave-csg5"
server_key: "4e2c3238a2807c037a52c39a6277064f"
server_key_iv: "e328e56335187a43c087e2f0fbbcde26"
server_key_version: "encryptionkey@0"
I20260812 07:08:28.051702 8314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:28.063730 8314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:28.068290 8314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:28.068507 8314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:28.069237 8314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:28.070314 8314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:28.070375 8314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:28.070429 8314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:28.070451 8314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:28.081678 8314 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:42147
I20260812 07:08:28.081701 8476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:42147 every 8 connection(s)
I20260812 07:08:28.082962 8314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:28.086365 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 8314
Aug 12 07:08:28 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518508, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.125@KRBTEST.COM
Aug 12 07:08:28 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518508, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.124@KRBTEST.COM
Aug 12 07:08:28 dist-test-slave-csg5 krb5kdc[6941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518508, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:28.101922 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.089439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.31.156.124:45641)
0812 07:08:28.089561 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.089565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.089721 (+ 156us) server_negotiation.cc:407] Connection header received
0812 07:08:28.089909 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.089913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.089965 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.090016 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:28.092190 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.092953 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.094156 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.094512 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.094583 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.096517 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:28.096521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:28.096524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:28.096552 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:28.099238 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.099788 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.099794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.099797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.099901 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.100252 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.100256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.100257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.100527 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:28.100644 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.101519 (+ 875us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.101698 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:08:28.105379 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.085249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:54291 (local address 127.31.156.126:41665)
0812 07:08:28.085615 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.085620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.088550 (+ 2930us) server_negotiation.cc:407] Connection header received
0812 07:08:28.088629 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.088632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.088700 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.088778 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:28.090915 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.091681 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.096949 (+ 5268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.097350 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.097475 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.100965 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:28.100971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:28.100976 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:28.101011 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:28.103216 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.103904 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.103907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.103909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.104012 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.104442 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.104445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.104446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.104626 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:28.104745 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.105065 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.105206 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274}
I20260812 07:08:28.105930 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.087419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46853 (local address 127.31.156.67:37135)
0812 07:08:28.090325 (+ 2906us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:28.090341 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:28.091026 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:28.091391 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:28.091394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:28.091580 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:28.091699 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.091706 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.093121 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.093127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.093976 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.093984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.095050 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.095054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.095430 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.096041 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:28.096057 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:28.097891 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:28.100211 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:28.100220 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:28.100243 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:28.100823 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:28.102617 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:28.102621 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:28.102623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:28.102880 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:28.103463 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:28.103476 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:28.103933 (+ 457us) client_negotiation.cc:769] Sending connection context
0812 07:08:28.104192 (+ 259us) client_negotiation.cc:240] Negotiation successful
0812 07:08:28.104627 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2819,"mutex_wait_us":1,"spinlock_wait_cycles":853248,"thread_start_us":101,"threads_started":1}
I20260812 07:08:28.105937 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.085797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.67:54291)
0812 07:08:28.088400 (+ 2603us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:28.088415 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:28.088534 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:28.090397 (+ 1863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:28.090400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:28.090606 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:28.090789 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.090796 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.094731 (+ 3935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.094735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.096782 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.096793 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.098125 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.098132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.098253 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.099201 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:28.099218 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:28.100827 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:28.103344 (+ 2517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:28.103348 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:28.103350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:28.103593 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:28.104175 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:28.104179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:28.104182 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:28.104289 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:28.104736 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:28.104739 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:28.104862 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 07:08:28.104981 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 07:08:28.105158 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2477,"mutex_wait_us":1205,"spinlock_wait_cycles":52096,"thread_start_us":175,"threads_started":1}
I20260812 07:08:28.107254 8491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.088871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34862 (local address 127.31.156.125:46853)
0812 07:08:28.091255 (+ 2384us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.091261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.091273 (+ 12us) server_negotiation.cc:407] Connection header received
0812 07:08:28.091309 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.091311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.091351 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.092713 (+ 1362us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:28.094852 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.095622 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.098288 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.100933 (+ 2645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.101078 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.103036 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:28.103039 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:28.103042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:28.103065 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:28.104889 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.105430 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.105433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.105435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.105527 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.105829 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.105833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.105834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.106067 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:28.106192 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.106982 (+ 790us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.107104 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2305,"thread_start_us":92,"threads_started":1}
I20260812 07:08:28.107617 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.087578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:37135 (local address 127.31.156.125:46853)
0812 07:08:28.087709 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.087714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.090426 (+ 2712us) server_negotiation.cc:407] Connection header received
0812 07:08:28.091128 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.091131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.091184 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.091265 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:28.091866 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.092698 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.094106 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.094823 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.094924 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.098100 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:28.098104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:28.098107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:28.098138 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:28.100071 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.101256 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.101260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.101262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.101773 (+ 511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.103019 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.103023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.103024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.103284 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:28.103392 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.106716 (+ 3324us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.106871 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:08:28.107718 8478 heartbeater.cc:347] Connected to a master server at 127.31.156.126:41665
I20260812 07:08:28.107991 8478 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:28.108570 8479 heartbeater.cc:347] Connected to a master server at 127.31.156.125:46853
I20260812 07:08:28.108667 8479 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:28.108702 8478 heartbeater.cc:511] Master 127.31.156.126:41665 requested a full tablet report, sending...
I20260812 07:08:28.108834 8479 heartbeater.cc:511] Master 127.31.156.125:46853 requested a full tablet report, sending...
I20260812 07:08:28.110041 7704 ts_manager.cc:194] Registered new tserver with Master: e2a1d23f5fed443ab9f448a0958ebbaa (127.31.156.67:42147)
I20260812 07:08:28.110178 6992 ts_manager.cc:194] Re-registered known tserver with Master: e2a1d23f5fed443ab9f448a0958ebbaa (127.31.156.67:42147)
I20260812 07:08:28.111387 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.089872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59782 (local address 127.31.156.126:41665)
0812 07:08:28.091572 (+ 1700us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.091586 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.091597 (+ 11us) server_negotiation.cc:407] Connection header received
0812 07:08:28.091635 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.091637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.091692 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.093768 (+ 2076us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:28.093946 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.094746 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.099896 (+ 5150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.100287 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.100398 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.102232 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:28.102237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:28.102243 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:28.102279 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:28.109071 (+ 6792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.109721 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.109725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.109727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.109829 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.110203 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.110206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.110218 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.110572 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:28.110759 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.111061 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.111202 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1624,"thread_start_us":65,"threads_started":1}
I20260812 07:08:28.111821 6992 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:54291
I20260812 07:08:28.113466 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.085432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:40105 (local address 127.31.156.124:45641)
0812 07:08:28.085581 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.085586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.087757 (+ 2171us) server_negotiation.cc:407] Connection header received
0812 07:08:28.088433 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.088441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.088511 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.088595 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:28.090365 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.091210 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.093200 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.093557 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.093636 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.108045 (+ 14409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:28.108049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:28.108052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:28.108086 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:28.111359 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.111910 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.111917 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.111919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.112022 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.112363 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.112368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.112370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.112611 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:28.112691 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.113190 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.113342 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:08:28.114225 8482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.085797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:45641 (local address 127.31.156.67:40105)
0812 07:08:28.087214 (+ 1417us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:28.087278 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:28.088179 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:28.088865 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:28.088875 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:28.089483 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:28.090205 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.090221 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.091390 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.091393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.093024 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.093034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.093848 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.093852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.094111 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.095465 (+ 1354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:28.095489 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:28.099099 (+ 3610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:28.111490 (+ 12391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:28.111493 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:28.111497 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:28.111774 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:28.112130 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:28.112133 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:28.112135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:28.112233 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:28.112775 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:28.112778 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:28.112953 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 07:08:28.113885 (+ 932us) client_negotiation.cc:240] Negotiation successful
0812 07:08:28.114040 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"mutex_wait_us":54,"thread_start_us":178,"threads_started":1}
I20260812 07:08:28.115016 8477 heartbeater.cc:347] Connected to a master server at 127.31.156.124:45641
I20260812 07:08:28.115124 8477 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:28.115332 8477 heartbeater.cc:511] Master 127.31.156.124:45641 requested a full tablet report, sending...
I20260812 07:08:28.116220 7931 ts_manager.cc:194] Registered new tserver with Master: e2a1d23f5fed443ab9f448a0958ebbaa (127.31.156.67:42147)
I20260812 07:08:28.114987 6991 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:08:28.118544 6991 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:28.121579 7892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: NON_VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: true } } }
I20260812 07:08:28.121774 7892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:28.124164 8496 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8: attempting to promote NON_VOTER 5b48d97e4ffa4a9589bb449878f0e118 to VOTER
I20260812 07:08:28.124678 8496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } }
I20260812 07:08:28.125990 7719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Refusing update from remote peer 0b9ef89962f743399485ce3efb4a26d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 07:08:28.126015 7943 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 LEARNER]: Refusing update from remote peer 0b9ef89962f743399485ce3efb4a26d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 07:08:28.126557 8496 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 07:08:28.126878 8497 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 07:08:28.128890 8497 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5b48d97e4ffa4a9589bb449878f0e118 (127.31.156.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:28.129395 7719 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5b48d97e4ffa4a9589bb449878f0e118 (127.31.156.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:28.129292 7943 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5b48d97e4ffa4a9589bb449878f0e118 (127.31.156.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:28.130625 7892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:28.130686 8497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:28.130817 7892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:28.130959 7892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:28.131042 7892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:28.131376 7978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:28.131366 8508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b9ef89962f743399485ce3efb4a26d8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:28.131489 7978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:28.131494 8508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:28.130833 8497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:28.145738 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.139668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:38327 (local address 127.0.0.1:34718)
0812 07:08:28.139820 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:28.139839 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:28.139928 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:28.140988 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:28.140991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:28.141031 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:28.141145 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.141152 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.142707 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.142710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.143917 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.143929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.145262 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.145266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.145364 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.145396 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:08:28.145470 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 07:08:28.145550 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"spinlock_wait_cycles":41728}
I20260812 07:08:28.145817 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.139769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.31.156.65:38327)
0812 07:08:28.140264 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.140297 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.140328 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:08:28.140427 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.140435 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.140808 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.141079 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:28.141258 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.142544 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.144078 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.145084 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.145263 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.145463 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.145530 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.145638 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":116,"threads_started":1}
I20260812 07:08:28.147296 8514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.139958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:41119 (local address 127.0.0.1:47094)
0812 07:08:28.140916 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:28.140931 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:28.141269 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:28.141936 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:28.141939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:28.141958 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:28.142076 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.142082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.143377 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.143383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.144493 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.144503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.146795 (+ 2292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.146804 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.146927 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.146954 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 07:08:28.147044 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 07:08:28.147131 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"thread_start_us":159,"threads_started":1}
I20260812 07:08:28.147789 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.140351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47094 (local address 127.31.156.66:41119)
0812 07:08:28.141508 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.141517 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.141539 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:08:28.141615 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.141619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.141804 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.141950 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:28.142215 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.143258 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.144634 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.145607 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.145706 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.145865 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.147488 (+ 1623us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.147628 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1055,"thread_start_us":119,"threads_started":1}
I20260812 07:08:28.149679 8511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.140537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:42147 (local address 127.0.0.1:56618)
0812 07:08:28.141461 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:28.141474 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:28.141554 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:28.142739 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:28.142742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:28.142762 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:28.142849 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.142855 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.146296 (+ 3441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.146300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.147658 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.147671 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.149186 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.149190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.149276 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.149380 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 07:08:28.149464 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 07:08:28.149544 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260812 07:08:28.149896 8515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.140738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56618 (local address 127.31.156.67:42147)
0812 07:08:28.142224 (+ 1486us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.142233 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.142251 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:08:28.142311 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.142315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.142455 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.142566 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:28.142989 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.144097 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.147869 (+ 3772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.149038 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.149186 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.149342 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.149512 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.149615 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1392,"thread_start_us":121,"threads_started":1}
I20260812 07:08:28.149448 8064 tablet_service.cc:1467] Processing CreateTablet for tablet 734eb8a1e1da4ab4950e63fa3c43226f (DEFAULT_TABLE table=test-table [id=26b6239d4ed84e4ab9ff8f228bfa05c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:28.150715 8064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734eb8a1e1da4ab4950e63fa3c43226f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:28.150518 8228 tablet_service.cc:1467] Processing CreateTablet for tablet 734eb8a1e1da4ab4950e63fa3c43226f (DEFAULT_TABLE table=test-table [id=26b6239d4ed84e4ab9ff8f228bfa05c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:28.151623 8228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734eb8a1e1da4ab4950e63fa3c43226f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:28.152549 8393 tablet_service.cc:1467] Processing CreateTablet for tablet 734eb8a1e1da4ab4950e63fa3c43226f (DEFAULT_TABLE table=test-table [id=26b6239d4ed84e4ab9ff8f228bfa05c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:28.153409 8393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734eb8a1e1da4ab4950e63fa3c43226f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:28.159889 8516 tablet_bootstrap.cc:492] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: Bootstrap starting.
I20260812 07:08:28.160722 8518 tablet_bootstrap.cc:492] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: Bootstrap starting.
I20260812 07:08:28.160830 8517 tablet_bootstrap.cc:492] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: Bootstrap starting.
I20260812 07:08:28.163169 8517 tablet_bootstrap.cc:654] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:28.163197 8518 tablet_bootstrap.cc:654] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:28.164036 8517 log.cc:824] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:28.164037 8518 log.cc:824] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:28.166621 8518 tablet_bootstrap.cc:492] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: No bootstrap required, opened a new log
I20260812 07:08:28.166695 8517 tablet_bootstrap.cc:492] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: No bootstrap required, opened a new log
I20260812 07:08:28.166919 8518 ts_tablet_manager.cc:1397] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 07:08:28.166919 8517 ts_tablet_manager.cc:1397] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260812 07:08:28.167513 8516 tablet_bootstrap.cc:654] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:28.168974 8516 log.cc:824] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:28.171288 8518 raft_consensus.cc:348] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } }
I20260812 07:08:28.171340 8517 raft_consensus.cc:348] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } }
I20260812 07:08:28.171587 8518 raft_consensus.cc:374] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:28.171620 8517 raft_consensus.cc:374] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:28.171650 8518 raft_consensus.cc:729] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a1d23f5fed443ab9f448a0958ebbaa, State: Initialized, Role: FOLLOWER
I20260812 07:08:28.171687 8517 raft_consensus.cc:729] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db6816cdaa914c2f8f70a15558a04a0e, State: Initialized, Role: FOLLOWER
I20260812 07:08:28.171743 8516 tablet_bootstrap.cc:492] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: No bootstrap required, opened a new log
I20260812 07:08:28.171973 8516 ts_tablet_manager.cc:1397] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260812 07:08:28.172250 8517 consensus_queue.cc:260] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } }
I20260812 07:08:28.172271 8518 consensus_queue.cc:260] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } }
I20260812 07:08:28.173480 8517 ts_tablet_manager.cc:1428] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 07:08:28.173481 8518 ts_tablet_manager.cc:1428] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260812 07:08:28.173687 8306 heartbeater.cc:503] Master 127.31.156.126:41665 was elected leader, sending a full tablet report...
I20260812 07:08:28.174710 8478 heartbeater.cc:503] Master 127.31.156.126:41665 was elected leader, sending a full tablet report...
I20260812 07:08:28.178637 8516 raft_consensus.cc:348] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } }
I20260812 07:08:28.179198 8516 raft_consensus.cc:374] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:28.179276 8516 raft_consensus.cc:729] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfeb604fa3744b96a91e3cca5065e8ba, State: Initialized, Role: FOLLOWER
I20260812 07:08:28.179877 8516 consensus_queue.cc:260] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } }
I20260812 07:08:28.180867 8145 heartbeater.cc:503] Master 127.31.156.126:41665 was elected leader, sending a full tablet report...
I20260812 07:08:28.182837 8516 ts_tablet_manager.cc:1428] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260812 07:08:28.285585 8524 raft_consensus.cc:482] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:28.285971 8524 raft_consensus.cc:504] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } }
I20260812 07:08:28.287463 8524 leader_election.cc:288] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db6816cdaa914c2f8f70a15558a04a0e (127.31.156.66:41119), e2a1d23f5fed443ab9f448a0958ebbaa (127.31.156.67:42147)
I20260812 07:08:28.292456 8526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.287805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:42147 (local address 127.31.156.65:50341)
0812 07:08:28.288330 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:28.288342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:28.288487 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:28.288811 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:28.288815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:28.288832 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:28.289002 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.289010 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.289894 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.289896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.291012 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.291024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.291876 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.291880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.291995 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.292016 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 07:08:28.292115 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 07:08:28.292212 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":108,"thread_start_us":152,"threads_started":1}
I20260812 07:08:28.292423 8515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.287932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:50341 (local address 127.31.156.67:42147)
0812 07:08:28.288094 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.288098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.288481 (+ 383us) server_negotiation.cc:407] Connection header received
0812 07:08:28.288633 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.288635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.288682 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.288780 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:28.289131 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.289739 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.291170 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.291713 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.291876 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.291940 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.292143 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.292227 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:08:28.292680 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.288038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:50539 (local address 127.31.156.66:41119)
0812 07:08:28.288194 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.288205 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.288420 (+ 215us) server_negotiation.cc:407] Connection header received
0812 07:08:28.288629 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.288632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.288680 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.288741 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:28.289165 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.289752 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.291544 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.291989 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.292140 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.292194 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.292456 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.292544 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:08:28.292739 8525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.287805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:41119 (local address 127.31.156.65:50539)
0812 07:08:28.288298 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:28.288311 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:28.288471 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:28.288789 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:28.288792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:28.288810 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:28.289009 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.289016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.289870 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.289876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.291375 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:28.291387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.292136 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.292147 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:28.292256 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.292311 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 07:08:28.292433 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 07:08:28.292516 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":87,"spinlock_wait_cycles":768,"thread_start_us":191,"threads_started":1}
I20260812 07:08:28.293521 8250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734eb8a1e1da4ab4950e63fa3c43226f" candidate_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db6816cdaa914c2f8f70a15558a04a0e" is_pre_election: true
I20260812 07:08:28.293538 8419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734eb8a1e1da4ab4950e63fa3c43226f" candidate_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" is_pre_election: true
I20260812 07:08:28.293939 8419 raft_consensus.cc:2454] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfeb604fa3744b96a91e3cca5065e8ba in term 0.
I20260812 07:08:28.293938 8250 raft_consensus.cc:2454] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfeb604fa3744b96a91e3cca5065e8ba in term 0.
I20260812 07:08:28.294564 8011 leader_election.cc:302] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: bfeb604fa3744b96a91e3cca5065e8ba, db6816cdaa914c2f8f70a15558a04a0e, e2a1d23f5fed443ab9f448a0958ebbaa; no voters:
I20260812 07:08:28.294963 8524 raft_consensus.cc:2781] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:28.295096 8524 raft_consensus.cc:482] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:28.295167 8524 raft_consensus.cc:3037] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:28.296437 8524 raft_consensus.cc:504] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } }
I20260812 07:08:28.296903 8524 leader_election.cc:288] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [CANDIDATE]: Term 1 election: Requested vote from peers db6816cdaa914c2f8f70a15558a04a0e (127.31.156.66:41119), e2a1d23f5fed443ab9f448a0958ebbaa (127.31.156.67:42147)
I20260812 07:08:28.297513 8250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734eb8a1e1da4ab4950e63fa3c43226f" candidate_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db6816cdaa914c2f8f70a15558a04a0e"
I20260812 07:08:28.297660 8250 raft_consensus.cc:3037] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:28.299520 8250 raft_consensus.cc:2454] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfeb604fa3744b96a91e3cca5065e8ba in term 1.
I20260812 07:08:28.299906 8419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734eb8a1e1da4ab4950e63fa3c43226f" candidate_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa"
I20260812 07:08:28.300040 8013 leader_election.cc:302] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfeb604fa3744b96a91e3cca5065e8ba, db6816cdaa914c2f8f70a15558a04a0e; no voters:
I20260812 07:08:28.300073 8419 raft_consensus.cc:3037] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:28.300333 8524 raft_consensus.cc:2781] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:28.300647 8524 raft_consensus.cc:686] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 1 LEADER]: Becoming Leader. State: Replica: bfeb604fa3744b96a91e3cca5065e8ba, State: Running, Role: LEADER
I20260812 07:08:28.301290 8524 consensus_queue.cc:237] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } }
I20260812 07:08:28.301883 8419 raft_consensus.cc:2454] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfeb604fa3744b96a91e3cca5065e8ba in term 1.
I20260812 07:08:28.305845 6992 catalog_manager.cc:5495] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba reported cstate change: term changed from 0 to 1, leader changed from <none> to bfeb604fa3744b96a91e3cca5065e8ba (127.31.156.65). New cstate: current_term: 1 leader_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } health_report { overall_health: UNKNOWN } } }
W20260812 07:08:28.334591 8480 tablet.cc:2369] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:08:28.337105 8148 tablet.cc:2369] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:08:28.340363 8309 tablet.cc:2369] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:08:28.342267 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.337941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.31.156.65:38327)
0812 07:08:28.338073 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.338077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.338092 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:08:28.338178 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.338181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.338237 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.338332 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:28.338743 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.339308 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.340366 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.340734 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.340879 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.341224 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:28.341845 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:28.341944 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.342080 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.342143 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:08:28.349825 8419 raft_consensus.cc:1261] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa [term 1 FOLLOWER]: Refusing update from remote peer bfeb604fa3744b96a91e3cca5065e8ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:28.349825 8250 raft_consensus.cc:1261] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e [term 1 FOLLOWER]: Refusing update from remote peer bfeb604fa3744b96a91e3cca5065e8ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:28.350812 8524 consensus_queue.cc:1035] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:28.351116 8527 consensus_queue.cc:1035] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:28.363336 8531 mvcc.cc:205] Tried to move back new op lower bound from 7317579810191556608 to 7317579810006114304. Current Snapshot: MvccSnapshot[applied={T|T < 7317579810191556608}]
I20260812 07:08:28.376330 6991 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:28.376557 6991 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:28.381699 6991 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b9ef89962f743399485ce3efb4a26d8: Sending DeleteTablet for 3 replicas of tablet 734eb8a1e1da4ab4950e63fa3c43226f
I20260812 07:08:28.382853 8228 tablet_service.cc:1514] Processing DeleteTablet for tablet 734eb8a1e1da4ab4950e63fa3c43226f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:28 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:47094
I20260812 07:08:28.382874 8064 tablet_service.cc:1514] Processing DeleteTablet for tablet 734eb8a1e1da4ab4950e63fa3c43226f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:28 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:34718
I20260812 07:08:28.383144 8393 tablet_service.cc:1514] Processing DeleteTablet for tablet 734eb8a1e1da4ab4950e63fa3c43226f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:28 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:56618
I20260812 07:08:28.383733 8544 tablet_replica.cc:330] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: stopping tablet replica
I20260812 07:08:28.383831 8543 tablet_replica.cc:330] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: stopping tablet replica
I20260812 07:08:28.384214 8544 raft_consensus.cc:2229] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:28.384258 8543 raft_consensus.cc:2229] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:28.384634 8544 raft_consensus.cc:2258] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:28.384734 8543 raft_consensus.cc:2258] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:28.386607 8544 ts_tablet_manager.cc:1905] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:28.387645 8543 ts_tablet_manager.cc:1905] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:28.388373 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 6956
I20260812 07:08:28.392612 8544 ts_tablet_manager.cc:1918] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:28.392745 8544 log.cc:1196] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/734eb8a1e1da4ab4950e63fa3c43226f
I20260812 07:08:28.393110 8544 ts_tablet_manager.cc:1939] T 734eb8a1e1da4ab4950e63fa3c43226f P e2a1d23f5fed443ab9f448a0958ebbaa: Deleting consensus metadata
I20260812 07:08:28.394075 8543 ts_tablet_manager.cc:1918] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:28.394198 8543 log.cc:1196] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-0/wal/wals/734eb8a1e1da4ab4950e63fa3c43226f
I20260812 07:08:28.394572 8543 ts_tablet_manager.cc:1939] T 734eb8a1e1da4ab4950e63fa3c43226f P bfeb604fa3744b96a91e3cca5065e8ba: Deleting consensus metadata
I20260812 07:08:28.395550 8542 tablet_replica.cc:330] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: stopping tablet replica
I20260812 07:08:28.395992 8542 raft_consensus.cc:2229] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:28.396384 8542 raft_consensus.cc:2258] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:28.398062 8542 ts_tablet_manager.cc:1905] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:28.401511 8542 ts_tablet_manager.cc:1918] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:28.401659 8542 log.cc:1196] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/734eb8a1e1da4ab4950e63fa3c43226f
I20260812 07:08:28.402061 8542 ts_tablet_manager.cc:1939] T 734eb8a1e1da4ab4950e63fa3c43226f P db6816cdaa914c2f8f70a15558a04a0e: Deleting consensus metadata
W20260812 07:08:28.404896 8176 connection.cc:535] server connection from 127.0.0.1:47094 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 07:08:28.404928 8332 connection.cc:535] server connection from 127.0.0.1:56618 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 07:08:28.404959 8012 connection.cc:535] server connection from 127.0.0.1:34718 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 07:08:28.409221 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.387012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.31.156.125:46853)
0812 07:08:28.387153 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.387158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.396965 (+ 9807us) server_negotiation.cc:407] Connection header received
0812 07:08:28.397041 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.397045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.397122 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.397239 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:28.398467 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.399549 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.401179 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.401830 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.401969 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.404768 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:28.404778 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:28.404786 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:28.404835 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:28.407133 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.407737 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.407741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.407743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.407862 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.408202 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.408206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.408208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.408429 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:28.408602 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.408897 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.409039 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:08:28.420210 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.385781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50070 (local address 127.31.156.124:45641)
0812 07:08:28.385934 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.385940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.409551 (+ 23611us) server_negotiation.cc:407] Connection header received
0812 07:08:28.409618 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.409621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.409690 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.409793 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:28.410703 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.411580 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.413039 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.413427 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.413528 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.415514 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:28.415520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:28.415523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:28.415559 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:28.417975 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.418505 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.418509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.418511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.418606 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.418928 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.418931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.418933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.419157 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:28.419301 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.419909 (+ 608us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.420035 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:08:28.422363 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.410939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.31.156.125:46853)
0812 07:08:28.411085 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.411090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.411868 (+ 778us) server_negotiation.cc:407] Connection header received
0812 07:08:28.411988 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.411991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.412041 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.412118 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:28.413814 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.414799 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.415969 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.416327 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.416406 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.418249 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:28.418254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:28.418259 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:28.418294 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:28.420327 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.420793 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.420799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.420802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.420898 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.421207 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.421210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.421212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.421467 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:28.421601 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.422011 (+ 410us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.422150 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:08:28.424508 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:28.411460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.31.156.124:45641)
0812 07:08:28.411614 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:28.411619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:28.412982 (+ 1363us) server_negotiation.cc:407] Connection header received
0812 07:08:28.413191 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:28.413196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:28.413355 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:28.413437 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:28.414581 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.415456 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.416602 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:28.417007 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:28.417091 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:28.418951 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:28.418958 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:28.418964 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:28.418998 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:28.422475 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.423057 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.423061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.423062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.423156 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:28.423488 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:28.423491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:28.423493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:28.423737 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:28.423908 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:28.424155 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 07:08:28.424300 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:08:29.369818 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.369247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.67:42171)
0812 07:08:29.369586 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.369688 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":96,"threads_started":1}
W20260812 07:08:29.370227 8478 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:41665 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
I20260812 07:08:29.371428 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.370929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.65:35127)
0812 07:08:29.371254 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.371341 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":88,"threads_started":1}
W20260812 07:08:29.371811 8145 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:41665 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
I20260812 07:08:29.385216 8556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.384708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.66:33907)
0812 07:08:29.385004 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.385096 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":118,"threads_started":1}
W20260812 07:08:29.385691 8306 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:41665 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
I20260812 07:08:29.882874 8557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0b9ef89962f743399485ce3efb4a26d8)
I20260812 07:08:29.882948 8559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0b9ef89962f743399485ce3efb4a26d8)
I20260812 07:08:29.883138 8557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } }
I20260812 07:08:29.883193 8559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } }
I20260812 07:08:29.884359 8557 leader_election.cc:288] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665), 5b48d97e4ffa4a9589bb449878f0e118 (127.31.156.124:45641)
I20260812 07:08:29.884975 8559 leader_election.cc:288] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665), 6f7dfd08c77349e598acda2b6757f016 (127.31.156.125:46853)
I20260812 07:08:29.885124 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.884583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60218)
0812 07:08:29.884896 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.884986 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":92,"threads_started":1}
W20260812 07:08:29.885478 7687 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
I20260812 07:08:29.885938 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.885006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60232)
0812 07:08:29.885749 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.885834 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":119,"threads_started":1}
W20260812 07:08:29.886299 7915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
I20260812 07:08:29.886685 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.886362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60246)
0812 07:08:29.886515 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.886573 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260812 07:08:29.886863 7687 leader_election.cc:334] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665): Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
I20260812 07:08:29.887672 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.887406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60254)
0812 07:08:29.887539 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.887599 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 07:08:29.887790 7915 leader_election.cc:334] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665): Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
I20260812 07:08:29.889590 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.884589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46853 (local address 127.0.0.1:34892)
0812 07:08:29.885414 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.885431 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:29.885557 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:29.885990 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:29.885994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:29.886015 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:29.886123 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:29.886130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:29.887396 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:29.887400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:29.888385 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:29.888396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:29.889152 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:29.889157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:29.889257 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:29.889273 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 07:08:29.889336 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 07:08:29.889445 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":70,"threads_started":1}
I20260812 07:08:29.889616 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.884605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:45641 (local address 127.0.0.1:50086)
0812 07:08:29.885271 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.885286 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:29.885451 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:29.885896 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:29.885899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:29.885919 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:29.886049 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:29.886057 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:29.887219 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:29.887223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:29.888435 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:29.888476 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:29.889095 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:29.889102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:29.889281 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:29.889309 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 07:08:29.889395 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 07:08:29.889458 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":39,"thread_start_us":85,"threads_started":1}
I20260812 07:08:29.889616 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.884874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34892 (local address 127.31.156.125:46853)
0812 07:08:29.885262 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:29.885269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:29.885656 (+ 387us) server_negotiation.cc:407] Connection header received
0812 07:08:29.885708 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:29.885712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:29.885768 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:29.885857 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:29.886272 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:29.887259 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:29.888522 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:29.888993 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:29.889144 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:29.889281 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:29.889365 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 07:08:29.889454 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":307,"thread_start_us":66,"threads_started":1}
I20260812 07:08:29.889590 8565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.884830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.31.156.124:45641)
0812 07:08:29.885181 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:29.885187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:29.885366 (+ 179us) server_negotiation.cc:407] Connection header received
0812 07:08:29.885591 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:29.885597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:29.885657 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:29.885730 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:29.886228 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:29.887051 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:29.888586 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:29.888968 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:29.889057 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:29.889144 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:29.889420 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 07:08:29.889483 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":76,"threads_started":1}
I20260812 07:08:29.890632 7943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f7dfd08c77349e598acda2b6757f016" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b48d97e4ffa4a9589bb449878f0e118" is_pre_election: true
I20260812 07:08:29.890920 7943 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f7dfd08c77349e598acda2b6757f016 in term 1.
I20260812 07:08:29.890867 7719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b48d97e4ffa4a9589bb449878f0e118" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f7dfd08c77349e598acda2b6757f016" is_pre_election: true
I20260812 07:08:29.891122 7719 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b48d97e4ffa4a9589bb449878f0e118 in term 1.
I20260812 07:08:29.891362 7686 leader_election.cc:302] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b48d97e4ffa4a9589bb449878f0e118, 6f7dfd08c77349e598acda2b6757f016; no voters: 0b9ef89962f743399485ce3efb4a26d8
I20260812 07:08:29.891625 8557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 07:08:29.891690 8557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0b9ef89962f743399485ce3efb4a26d8)
I20260812 07:08:29.891746 8557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:08:29.891960 7914 leader_election.cc:302] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b48d97e4ffa4a9589bb449878f0e118, 6f7dfd08c77349e598acda2b6757f016; no voters: 0b9ef89962f743399485ce3efb4a26d8
I20260812 07:08:29.892309 8559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 07:08:29.892381 8559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0b9ef89962f743399485ce3efb4a26d8)
I20260812 07:08:29.892411 8559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:08:29.893137 8557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } }
I20260812 07:08:29.893493 8559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } }
I20260812 07:08:29.893627 8557 leader_election.cc:288] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 2 election: Requested vote from peers 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665), 5b48d97e4ffa4a9589bb449878f0e118 (127.31.156.124:45641)
I20260812 07:08:29.893915 8559 leader_election.cc:288] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [CANDIDATE]: Term 2 election: Requested vote from peers 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665), 6f7dfd08c77349e598acda2b6757f016 (127.31.156.125:46853)
I20260812 07:08:29.894059 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.893749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60262)
0812 07:08:29.893895 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.893972 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 07:08:29.894122 7943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f7dfd08c77349e598acda2b6757f016" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b48d97e4ffa4a9589bb449878f0e118"
I20260812 07:08:29.894289 7943 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f7dfd08c77349e598acda2b6757f016 in current term 2: Already voted for candidate 5b48d97e4ffa4a9589bb449878f0e118 in this term.
I20260812 07:08:29.894234 7719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b48d97e4ffa4a9589bb449878f0e118" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f7dfd08c77349e598acda2b6757f016"
I20260812 07:08:29.894474 7719 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b48d97e4ffa4a9589bb449878f0e118 in current term 2: Already voted for candidate 6f7dfd08c77349e598acda2b6757f016 in this term.
I20260812 07:08:29.894964 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.894049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60274)
0812 07:08:29.894833 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.894899 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":693}
I20260812 07:08:29.895567 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.895266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60282)
0812 07:08:29.895424 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.895490 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 07:08:29.895838 7687 leader_election.cc:334] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665): Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
I20260812 07:08:29.895938 7687 leader_election.cc:302] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f7dfd08c77349e598acda2b6757f016; no voters: 0b9ef89962f743399485ce3efb4a26d8, 5b48d97e4ffa4a9589bb449878f0e118
I20260812 07:08:29.896134 8557 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 07:08:29.896517 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:29.895510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60296)
0812 07:08:29.896381 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:29.896450 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 07:08:29.896766 7915 leader_election.cc:334] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665): Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
I20260812 07:08:29.896853 7915 leader_election.cc:302] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b48d97e4ffa4a9589bb449878f0e118; no voters: 0b9ef89962f743399485ce3efb4a26d8, 6f7dfd08c77349e598acda2b6757f016
I20260812 07:08:29.897023 8559 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 07:08:30.371842 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:30.371104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.67:41265)
0812 07:08:30.371516 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:30.371608 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":152,"threads_started":1}
I20260812 07:08:30.373337 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:30.372434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.65:35369)
0812 07:08:30.372946 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:30.373102 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":114,"threads_started":1}
I20260812 07:08:30.387149 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:30.386589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.66:42887)
0812 07:08:30.386968 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:30.387039 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":157,"threads_started":1}
I20260812 07:08:31.373857 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.373356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.67:54763)
0812 07:08:31.373636 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.373725 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
I20260812 07:08:31.374646 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.374071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.65:54625)
0812 07:08:31.374479 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.374558 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":75,"threads_started":1}
I20260812 07:08:31.388616 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.388112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.66:58125)
0812 07:08:31.388440 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.388509 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":97,"threads_started":1}
I20260812 07:08:31.465593 8577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:31.465822 8577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } }
I20260812 07:08:31.466391 8577 leader_election.cc:288] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665), 5b48d97e4ffa4a9589bb449878f0e118 (127.31.156.124:45641)
I20260812 07:08:31.467077 7943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f7dfd08c77349e598acda2b6757f016" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b48d97e4ffa4a9589bb449878f0e118" is_pre_election: true
I20260812 07:08:31.467306 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.466816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60514)
0812 07:08:31.467097 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.467158 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":112,"threads_started":1}
I20260812 07:08:31.467260 7943 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f7dfd08c77349e598acda2b6757f016 in term 2.
I20260812 07:08:31.467799 7686 leader_election.cc:302] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b48d97e4ffa4a9589bb449878f0e118, 6f7dfd08c77349e598acda2b6757f016; no voters:
I20260812 07:08:31.468044 8577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 07:08:31.468145 8577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:31.468212 8577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 2 FOLLOWER]: Advancing to term 3
I20260812 07:08:31.468321 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.468032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60528)
0812 07:08:31.468149 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.468228 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 07:08:31.468454 7687 leader_election.cc:334] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665): Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
I20260812 07:08:31.469506 8577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } }
I20260812 07:08:31.469919 8577 leader_election.cc:288] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 3 election: Requested vote from peers 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665), 5b48d97e4ffa4a9589bb449878f0e118 (127.31.156.124:45641)
I20260812 07:08:31.470402 7943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f7dfd08c77349e598acda2b6757f016" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b48d97e4ffa4a9589bb449878f0e118"
I20260812 07:08:31.470479 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.470136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60540)
0812 07:08:31.470339 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.470399 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260812 07:08:31.470559 7943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 2 FOLLOWER]: Advancing to term 3
I20260812 07:08:31.471380 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.471098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60554)
0812 07:08:31.471256 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.471314 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 07:08:31.471522 7687 leader_election.cc:334] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665): Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
I20260812 07:08:31.471974 7943 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f7dfd08c77349e598acda2b6757f016 in term 3.
I20260812 07:08:31.472322 7686 leader_election.cc:302] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b48d97e4ffa4a9589bb449878f0e118, 6f7dfd08c77349e598acda2b6757f016; no voters: 0b9ef89962f743399485ce3efb4a26d8
I20260812 07:08:31.472584 8577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 07:08:31.472834 8577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [term 3 LEADER]: Becoming Leader. State: Replica: 6f7dfd08c77349e598acda2b6757f016, State: Running, Role: LEADER
I20260812 07:08:31.473140 8577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } }
I20260812 07:08:31.475188 8581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f7dfd08c77349e598acda2b6757f016. Latest consensus state: current_term: 3 leader_uuid: "6f7dfd08c77349e598acda2b6757f016" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:31.475384 8581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:31.475833 8584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:31.478356 8584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:31.478847 8584 catalog_manager.cc:1259] Loaded cluster ID: f8f3870a34c84e4791cfae7eac9fbe99
I20260812 07:08:31.478920 8584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:31.479503 8584 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:31.479907 8584 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: Loaded TSK: 0
I20260812 07:08:31.480615 8584 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:31.513751 7705 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:08:31.517427 7705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:31.521241 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.520832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60584)
0812 07:08:31.521081 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.521159 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260812 07:08:31.521445 7943 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [term 3 FOLLOWER]: Refusing update from remote peer 6f7dfd08c77349e598acda2b6757f016: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 07:08:31.522060 8581 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 07:08:31.522203 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.521956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.0.0.1:60600)
0812 07:08:31.522078 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.522138 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:08:31.522388 7687 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 -> Peer 0b9ef89962f743399485ce3efb4a26d8 (127.31.156.126:41665): Couldn't send request to peer 0b9ef89962f743399485ce3efb4a26d8. Status: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 07:08:31.524030 8587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f7dfd08c77349e598acda2b6757f016. Latest consensus state: current_term: 3 leader_uuid: "6f7dfd08c77349e598acda2b6757f016" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:31.524170 8587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:31.525463 8585 mvcc.cc:205] Tried to move back new op lower bound from 7317579823184670720 to 7317579822997585920. Current Snapshot: MvccSnapshot[applied={T|T < 7317579823184670720}]
I20260812 07:08:31.525471 8577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "6f7dfd08c77349e598acda2b6757f016" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:31.525648 8577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:31.525971 8587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "6f7dfd08c77349e598acda2b6757f016" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0b9ef89962f743399485ce3efb4a26d8" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 41665 } } peers { permanent_uuid: "6f7dfd08c77349e598acda2b6757f016" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46853 } attrs { promote: false } } peers { permanent_uuid: "5b48d97e4ffa4a9589bb449878f0e118" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 45641 } attrs { promote: false } } }
I20260812 07:08:31.526108 8587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b48d97e4ffa4a9589bb449878f0e118 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:31.538883 8595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.533094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34740 (local address 127.31.156.65:38327)
0812 07:08:31.533922 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:31.533929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:31.533947 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:08:31.534009 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:31.534012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:31.534064 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:31.534206 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:31.534818 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.535798 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.537643 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.538180 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.538331 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:31.538409 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:31.538617 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 07:08:31.538737 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":127,"threads_started":1}
I20260812 07:08:31.538883 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.532992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:38327 (local address 127.0.0.1:34740)
0812 07:08:31.533180 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.533201 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:31.533325 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:31.534222 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:31.534255 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:31.534406 (+ 151us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:31.534602 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:31.534610 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.535988 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.535992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:31.537470 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:31.537482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.538338 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.538342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:31.538449 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:31.538475 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 07:08:31.538582 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 07:08:31.538705 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":91}
I20260812 07:08:31.539321 8597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.533457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:42147 (local address 127.0.0.1:56626)
0812 07:08:31.534403 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.534414 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:31.534532 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:31.535036 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:31.535039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:31.535056 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:31.535137 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:31.535143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.536412 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.536415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:31.537268 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:31.537278 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.538626 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.538630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:31.538778 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:31.538808 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:08:31.538910 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 07:08:31.538992 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"mutex_wait_us":8,"spinlock_wait_cycles":2432,"thread_start_us":106,"threads_started":1}
I20260812 07:08:31.539358 8598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.533843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56626 (local address 127.31.156.67:42147)
0812 07:08:31.534258 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:31.534262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:31.534616 (+ 354us) server_negotiation.cc:407] Connection header received
0812 07:08:31.534707 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:31.534710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:31.534771 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:31.534907 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:31.535269 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.535993 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.537529 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.538352 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.538481 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:31.538563 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:31.538934 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 07:08:31.539205 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":87,"threads_started":1}
I20260812 07:08:31.539999 8393 tablet_service.cc:1467] Processing CreateTablet for tablet 5cc03505ebe04600ae46dfb2dcaca357 (DEFAULT_TABLE table=test-table [id=98c5ce3f9ad24796b1dd1a8d1de9884d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:31.540108 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.533450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:41119 (local address 127.0.0.1:47098)
0812 07:08:31.533811 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.533828 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:31.533933 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:31.534505 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:31.534508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:31.534533 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:31.534690 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:31.534697 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.536182 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.536189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:31.537609 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:31.537619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.539691 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.539695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:31.539798 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:31.539823 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:08:31.539881 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 07:08:31.539955 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":30,"spinlock_wait_cycles":3200,"thread_start_us":115,"threads_started":1}
I20260812 07:08:31.540138 8064 tablet_service.cc:1467] Processing CreateTablet for tablet 5cc03505ebe04600ae46dfb2dcaca357 (DEFAULT_TABLE table=test-table [id=98c5ce3f9ad24796b1dd1a8d1de9884d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:31.540392 8393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cc03505ebe04600ae46dfb2dcaca357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:31.540469 8064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cc03505ebe04600ae46dfb2dcaca357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:31.540683 8599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.533912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47098 (local address 127.31.156.66:41119)
0812 07:08:31.534225 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:31.534230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:31.534247 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:08:31.534302 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:31.534305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:31.534360 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:31.534465 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:31.534908 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.536028 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.537752 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.538494 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.540316 (+ 1822us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:31.540394 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:31.540442 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 07:08:31.540531 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":133,"threads_started":1}
I20260812 07:08:31.542974 8601 tablet_bootstrap.cc:492] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa: Bootstrap starting.
I20260812 07:08:31.543450 8228 tablet_service.cc:1467] Processing CreateTablet for tablet 5cc03505ebe04600ae46dfb2dcaca357 (DEFAULT_TABLE table=test-table [id=98c5ce3f9ad24796b1dd1a8d1de9884d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:31.543758 8228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cc03505ebe04600ae46dfb2dcaca357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:31.544098 8603 tablet_bootstrap.cc:492] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba: Bootstrap starting.
I20260812 07:08:31.544202 8601 tablet_bootstrap.cc:654] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:31.545349 8603 tablet_bootstrap.cc:654] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:31.545783 8601 tablet_bootstrap.cc:492] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa: No bootstrap required, opened a new log
I20260812 07:08:31.545897 8601 ts_tablet_manager.cc:1397] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:31.546473 8601 raft_consensus.cc:348] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } }
I20260812 07:08:31.546641 8601 raft_consensus.cc:374] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:31.546759 8601 raft_consensus.cc:729] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a1d23f5fed443ab9f448a0958ebbaa, State: Initialized, Role: FOLLOWER
I20260812 07:08:31.546957 8603 tablet_bootstrap.cc:492] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba: No bootstrap required, opened a new log
I20260812 07:08:31.547070 8603 ts_tablet_manager.cc:1397] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:31.546980 8601 consensus_queue.cc:260] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } }
I20260812 07:08:31.547112 8606 tablet_bootstrap.cc:492] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e: Bootstrap starting.
I20260812 07:08:31.547559 8601 ts_tablet_manager.cc:1428] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 07:08:31.547616 8603 raft_consensus.cc:348] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } }
I20260812 07:08:31.547772 8603 raft_consensus.cc:374] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:31.547829 8603 raft_consensus.cc:729] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfeb604fa3744b96a91e3cca5065e8ba, State: Initialized, Role: FOLLOWER
I20260812 07:08:31.547993 8603 consensus_queue.cc:260] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } }
I20260812 07:08:31.548527 8606 tablet_bootstrap.cc:654] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:31.548522 8603 ts_tablet_manager.cc:1428] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 07:08:31.549636 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.548696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.67:52861)
0812 07:08:31.549477 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.549549 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":680}
I20260812 07:08:31.549635 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.549347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.65:38535)
0812 07:08:31.549472 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.549542 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 07:08:31.551046 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.550635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.67:50341)
0812 07:08:31.550896 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.550966 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260812 07:08:31.551280 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.550949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.65:37195)
0812 07:08:31.551141 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.551203 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260812 07:08:31.552172 8606 tablet_bootstrap.cc:492] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e: No bootstrap required, opened a new log
I20260812 07:08:31.552295 8606 ts_tablet_manager.cc:1397] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 07:08:31.552635 8479 heartbeater.cc:503] Master 127.31.156.125:46853 was elected leader, sending a full tablet report...
I20260812 07:08:31.552911 8606 raft_consensus.cc:348] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } }
I20260812 07:08:31.553048 8606 raft_consensus.cc:374] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:31.553085 8606 raft_consensus.cc:729] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db6816cdaa914c2f8f70a15558a04a0e, State: Initialized, Role: FOLLOWER
I20260812 07:08:31.553239 8606 consensus_queue.cc:260] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } }
I20260812 07:08:31.553495 8147 heartbeater.cc:503] Master 127.31.156.125:46853 was elected leader, sending a full tablet report...
I20260812 07:08:31.553769 8606 ts_tablet_manager.cc:1428] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:08:31.554993 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.554638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.66:54559)
0812 07:08:31.554851 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.554912 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260812 07:08:31.662617 8609 raft_consensus.cc:482] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:31.662868 8609 raft_consensus.cc:504] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } }
I20260812 07:08:31.663470 8609 leader_election.cc:288] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a1d23f5fed443ab9f448a0958ebbaa (127.31.156.67:42147), db6816cdaa914c2f8f70a15558a04a0e (127.31.156.66:41119)
I20260812 07:08:31.663988 8250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc03505ebe04600ae46dfb2dcaca357" candidate_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db6816cdaa914c2f8f70a15558a04a0e" is_pre_election: true
I20260812 07:08:31.663996 8419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc03505ebe04600ae46dfb2dcaca357" candidate_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" is_pre_election: true
I20260812 07:08:31.664175 8250 raft_consensus.cc:2454] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfeb604fa3744b96a91e3cca5065e8ba in term 0.
I20260812 07:08:31.664198 8419 raft_consensus.cc:2454] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfeb604fa3744b96a91e3cca5065e8ba in term 0.
I20260812 07:08:31.664584 8011 leader_election.cc:302] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfeb604fa3744b96a91e3cca5065e8ba, e2a1d23f5fed443ab9f448a0958ebbaa; no voters:
I20260812 07:08:31.664801 8609 raft_consensus.cc:2781] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:31.664872 8609 raft_consensus.cc:482] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:31.664911 8609 raft_consensus.cc:3037] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:31.666075 8609 raft_consensus.cc:504] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } }
I20260812 07:08:31.666591 8609 leader_election.cc:288] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [CANDIDATE]: Term 1 election: Requested vote from peers e2a1d23f5fed443ab9f448a0958ebbaa (127.31.156.67:42147), db6816cdaa914c2f8f70a15558a04a0e (127.31.156.66:41119)
I20260812 07:08:31.667088 8419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc03505ebe04600ae46dfb2dcaca357" candidate_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa"
I20260812 07:08:31.667271 8419 raft_consensus.cc:3037] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:31.667234 8250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc03505ebe04600ae46dfb2dcaca357" candidate_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db6816cdaa914c2f8f70a15558a04a0e"
I20260812 07:08:31.667415 8250 raft_consensus.cc:3037] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:31.668426 8250 raft_consensus.cc:2454] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfeb604fa3744b96a91e3cca5065e8ba in term 1.
I20260812 07:08:31.668431 8419 raft_consensus.cc:2454] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfeb604fa3744b96a91e3cca5065e8ba in term 1.
I20260812 07:08:31.668814 8011 leader_election.cc:302] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: bfeb604fa3744b96a91e3cca5065e8ba, db6816cdaa914c2f8f70a15558a04a0e, e2a1d23f5fed443ab9f448a0958ebbaa; no voters:
I20260812 07:08:31.668999 8609 raft_consensus.cc:2781] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:31.669229 8609 raft_consensus.cc:686] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 1 LEADER]: Becoming Leader. State: Replica: bfeb604fa3744b96a91e3cca5065e8ba, State: Running, Role: LEADER
I20260812 07:08:31.669379 8609 consensus_queue.cc:237] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } }
I20260812 07:08:31.670197 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.669889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.65:37877)
0812 07:08:31.670023 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.670101 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 07:08:31.671645 7705 catalog_manager.cc:5495] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba reported cstate change: term changed from 0 to 1, leader changed from <none> to bfeb604fa3744b96a91e3cca5065e8ba (127.31.156.65). New cstate: current_term: 1 leader_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfeb604fa3744b96a91e3cca5065e8ba" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 38327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 } health_report { overall_health: UNKNOWN } } }
I20260812 07:08:31.699290 8595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.695417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.31.156.65:38327)
0812 07:08:31.695576 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:31.695580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:31.695621 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:08:31.695676 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:31.695679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:31.695737 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:31.695801 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:31.696069 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.696684 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.697716 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:31.697948 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:31.698043 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:31.698396 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:31.698491 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:31.698568 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:31.698882 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 07:08:31.698958 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:08:31.702194 8419 raft_consensus.cc:1261] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa [term 1 FOLLOWER]: Refusing update from remote peer bfeb604fa3744b96a91e3cca5065e8ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:31.702262 8250 raft_consensus.cc:1261] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e [term 1 FOLLOWER]: Refusing update from remote peer bfeb604fa3744b96a91e3cca5065e8ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:31.702826 8614 consensus_queue.cc:1035] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a1d23f5fed443ab9f448a0958ebbaa" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:31.703025 8609 consensus_queue.cc:1035] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "db6816cdaa914c2f8f70a15558a04a0e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:31.704293 8308 heartbeater.cc:503] Master 127.31.156.125:46853 was elected leader, sending a full tablet report...
I20260812 07:08:31.705272 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.704770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.66:53767)
0812 07:08:31.705096 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.705181 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250}
I20260812 07:08:31.706149 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.705674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.67:42863)
0812 07:08:31.705986 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.706061 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234}
I20260812 07:08:31.708938 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.708620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.65:46913)
0812 07:08:31.708776 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.708837 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 07:08:31.711150 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.710824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.66:35045)
0812 07:08:31.710992 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.711056 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 07:08:31.711102 8619 mvcc.cc:205] Tried to move back new op lower bound from 7317579823927111680 to 7317579823799709696. Current Snapshot: MvccSnapshot[applied={T|T < 7317579823927111680}]
I20260812 07:08:31.712248 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.711675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.65:55615)
0812 07:08:31.712092 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.712160 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335}
I20260812 07:08:31.713784 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.713075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:41665 (local address 127.31.156.67:34267)
0812 07:08:31.713601 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.713688 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423}
I20260812 07:08:31.714021 7703 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:31.714251 7703 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:31.715412 8618 mvcc.cc:205] Tried to move back new op lower bound from 7317579823927111680 to 7317579823799709696. Current Snapshot: MvccSnapshot[applied={T|T < 7317579823927111680 or (T in {7317579823927111680})}]
I20260812 07:08:31.720570 7703 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f7dfd08c77349e598acda2b6757f016: Sending DeleteTablet for 3 replicas of tablet 5cc03505ebe04600ae46dfb2dcaca357
I20260812 07:08:31.721493 8393 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cc03505ebe04600ae46dfb2dcaca357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:31 UTC) from {username='oryx', principal='oryx/127.31.156.125@KRBTEST.COM'} at 127.0.0.1:56626
I20260812 07:08:31.721566 8228 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cc03505ebe04600ae46dfb2dcaca357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:31 UTC) from {username='oryx', principal='oryx/127.31.156.125@KRBTEST.COM'} at 127.0.0.1:47098
I20260812 07:08:31.721822 8064 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cc03505ebe04600ae46dfb2dcaca357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:31 UTC) from {username='oryx', principal='oryx/127.31.156.125@KRBTEST.COM'} at 127.0.0.1:34740
I20260812 07:08:31.722038 8063 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cc03505ebe04600ae46dfb2dcaca357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:31 UTC) from {username='oryx', principal='oryx/127.31.156.125@KRBTEST.COM'} at 127.0.0.1:34740
I20260812 07:08:31.722504 8627 tablet_replica.cc:330] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa: stopping tablet replica
I20260812 07:08:31.722522 8628 tablet_replica.cc:330] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e: stopping tablet replica
I20260812 07:08:31.722721 8628 raft_consensus.cc:2229] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:31.722731 8627 raft_consensus.cc:2229] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:31.722924 8628 raft_consensus.cc:2258] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:31.722923 8627 raft_consensus.cc:2258] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:31.723829 8628 ts_tablet_manager.cc:1905] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:31.724392 8629 tablet_replica.cc:330] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba: stopping tablet replica
I20260812 07:08:31.724540 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 7878
I20260812 07:08:31.724612 8629 raft_consensus.cc:2229] T 5cc03505ebe04600ae46dfb2dcaca357 P bfeb604fa3744b96a91e3cca5065e8ba [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:31.725636 8627 ts_tablet_manager.cc:1905] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:31.728260 8627 ts_tablet_manager.cc:1918] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:31.728343 8627 log.cc:1196] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/5cc03505ebe04600ae46dfb2dcaca357
I20260812 07:08:31.728598 8627 ts_tablet_manager.cc:1939] T 5cc03505ebe04600ae46dfb2dcaca357 P e2a1d23f5fed443ab9f448a0958ebbaa: Deleting consensus metadata
I20260812 07:08:31.729825 7686 catalog_manager.cc:4841] TS e2a1d23f5fed443ab9f448a0958ebbaa (127.31.156.67:42147): tablet 5cc03505ebe04600ae46dfb2dcaca357 (table test-table [id=98c5ce3f9ad24796b1dd1a8d1de9884d]) successfully deleted
I20260812 07:08:31.731426 8628 ts_tablet_manager.cc:1918] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:31.731550 8628 log.cc:1196] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/5cc03505ebe04600ae46dfb2dcaca357
I20260812 07:08:31.731818 8628 ts_tablet_manager.cc:1939] T 5cc03505ebe04600ae46dfb2dcaca357 P db6816cdaa914c2f8f70a15558a04a0e: Deleting consensus metadata
I20260812 07:08:31.733139 7688 catalog_manager.cc:4841] TS db6816cdaa914c2f8f70a15558a04a0e (127.31.156.66:41119): tablet 5cc03505ebe04600ae46dfb2dcaca357 (table test-table [id=98c5ce3f9ad24796b1dd1a8d1de9884d]) successfully deleted
I20260812 07:08:31.737636 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 8154
I20260812 07:08:31.738029 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:31.737647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:38327 (local address 127.0.0.1:34762)
0812 07:08:31.737841 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:31.737912 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260812 07:08:31.739681 7688 catalog_manager.cc:4584] TS bfeb604fa3744b96a91e3cca5065e8ba (127.31.156.65:38327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5cc03505ebe04600ae46dfb2dcaca357: Network error: Client connection negotiation failed: client connection to 127.31.156.65:38327: connect: Connection refused (error 111)
I20260812 07:08:31.747581 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 8314
I20260812 07:08:31.758936 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 7546
I20260812 07:08:31.768893 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 7660
2026-08-12T07:08:31Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7777 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:31 dist-test-slave-csg5 krb5kdc[8632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:31 dist-test-slave-csg5 krb5kdc[8632](info): set up 2 sockets
Aug 12 07:08:31 dist-test-slave-csg5 krb5kdc[8632](info): commencing operation
krb5kdc: starting...
W20260812 07:08:32.967566 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:08:33 dist-test-slave-csg5 krb5kdc[8632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.126.keytab.
Entry for principal oryx/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.126.keytab.
I20260812 07:08:33.128753 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:37337
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.126.keytab
--principal=oryx/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:37337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:33.251509 8647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:33.251852 8647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:33.251897 8647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:33.256247 8647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:33.256345 8647 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 07:08:33.256363 8647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:33.256382 8647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:33.256400 8647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:33.256417 8647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:33.262313 8647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:37337
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.126.keytab
--principal=oryx/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:37337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:33.263635 8647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:33.264801 8647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:33.275038 8656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:33.275035 8654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:33.275035 8653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:33.275679 8647 server_base.cc:1034] running on GCE node
I20260812 07:08:33.276616 8647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:33.278007 8647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:33.279192 8647 hybrid_clock.cc:648] HybridClock initialized: now 1786518513279154 us; error 56 us; skew 500 ppm
Aug 12 07:08:33 dist-test-slave-csg5 krb5kdc[8632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518513, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:33.289996 8647 init.cc:377] Logged in from keytab as oryx/127.31.156.126@KRBTEST.COM (short username oryx)
I20260812 07:08:33.291285 8647 webserver.cc:460] Webserver started at http://127.31.156.126:41447/ using document root <none> and password file <none>
I20260812 07:08:33.291951 8647 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:33.292027 8647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:33.292219 8647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:33.294605 8647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "eb677f7872c640c69cc53f045ffd73d2"
format_stamp: "Formatted at 2026-08-12 07:08:33 on dist-test-slave-csg5"
server_key: "3c9859a5e873bfc3d2f68e36081d3cbc"
server_key_iv: "d82effe6ce657b601dad8518b862d89f"
server_key_version: "encryptionkey@0"
I20260812 07:08:33.295389 8647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "eb677f7872c640c69cc53f045ffd73d2"
format_stamp: "Formatted at 2026-08-12 07:08:33 on dist-test-slave-csg5"
server_key: "3c9859a5e873bfc3d2f68e36081d3cbc"
server_key_iv: "d82effe6ce657b601dad8518b862d89f"
server_key_version: "encryptionkey@0"
I20260812 07:08:33.300161 8647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:08:33.303524 8663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:33.305552 8647 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:08:33.305739 8647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "eb677f7872c640c69cc53f045ffd73d2"
format_stamp: "Formatted at 2026-08-12 07:08:33 on dist-test-slave-csg5"
server_key: "3c9859a5e873bfc3d2f68e36081d3cbc"
server_key_iv: "d82effe6ce657b601dad8518b862d89f"
server_key_version: "encryptionkey@0"
I20260812 07:08:33.305927 8647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:33.323280 8647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:33.327597 8647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:33.327874 8647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:33.338155 8647 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:37337
I20260812 07:08:33.338187 8732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:37337 every 8 connection(s)
I20260812 07:08:33.339380 8647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:33.342470 8733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:33.348486 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 8647
I20260812 07:08:33.348668 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:33.348963 8733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2: Bootstrap starting.
I20260812 07:08:33.349016 32369 external_mini_cluster.cc:1428] Setting key 16b2738fc25995e9f8dca41c22371696
I20260812 07:08:33.351815 8733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:33.352742 8733 log.cc:824] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:33.355178 8733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2: No bootstrap required, opened a new log
Aug 12 07:08:33 dist-test-slave-csg5 krb5kdc[8632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:33.359045 8733 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb677f7872c640c69cc53f045ffd73d2" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37337 } }
I20260812 07:08:33.359469 8733 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:33.359561 8733 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb677f7872c640c69cc53f045ffd73d2, State: Initialized, Role: FOLLOWER
I20260812 07:08:33.360143 8733 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb677f7872c640c69cc53f045ffd73d2" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37337 } }
I20260812 07:08:33.360359 8733 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:33.360450 8733 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:33.360590 8733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:33.362016 8733 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb677f7872c640c69cc53f045ffd73d2" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37337 } }
I20260812 07:08:33.362465 8733 leader_election.cc:302] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb677f7872c640c69cc53f045ffd73d2; no voters:
I20260812 07:08:33.362936 8733 leader_election.cc:288] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:33.363013 8739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:33.363224 8739 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [term 1 LEADER]: Becoming Leader. State: Replica: eb677f7872c640c69cc53f045ffd73d2, State: Running, Role: LEADER
I20260812 07:08:33.363561 8739 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb677f7872c640c69cc53f045ffd73d2" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37337 } }
I20260812 07:08:33.364427 8733 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:33.364876 8739 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb677f7872c640c69cc53f045ffd73d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb677f7872c640c69cc53f045ffd73d2" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37337 } } }
I20260812 07:08:33.364995 8739 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:33.365216 8739 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb677f7872c640c69cc53f045ffd73d2. Latest consensus state: current_term: 1 leader_uuid: "eb677f7872c640c69cc53f045ffd73d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb677f7872c640c69cc53f045ffd73d2" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37337 } } }
I20260812 07:08:33.365309 8739 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:33.366379 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:33.350617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58340 (local address 127.31.156.126:37337)
0812 07:08:33.351323 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:33.351341 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:33.351382 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:08:33.352176 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:33.352205 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:33.352575 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:33.352927 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:33.353974 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.355193 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.356211 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.356764 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.356960 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:33.360081 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:33.360089 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:33.360108 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:33.360147 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:33.362518 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:33.363351 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:33.363362 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:33.363370 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:33.363523 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:33.363926 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:33.363930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:33.363932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:33.364393 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:33.364617 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:33.364955 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 07:08:33.365346 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":184,"threads_started":1}
W20260812 07:08:33.369562 8758 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:08:33.369671 8758 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:08:33.369750 8744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:33.370523 8744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:33.376206 8744 catalog_manager.cc:1347] Generated new cluster ID: bb0513102ad04e8b9c16752d339233f6
I20260812 07:08:33.376309 8744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:33.389348 8744 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:33.390832 8744 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:33.397081 8744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2: Generated new TSK 0
I20260812 07:08:33.397953 8744 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab.
Entry for principal oryx/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab.
I20260812 07:08:33.444154 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:37337
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab
--principal=oryx/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:33.579614 8764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:33.579964 8764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:33.580021 8764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:33.585508 8764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:33.585631 8764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:33.585682 8764 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:08:33.585722 8764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:33.592054 8764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab
--principal=oryx/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.156.126:37337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:33.593386 8764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:33.594586 8764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:33.606911 8770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:33.607017 8773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:33.607033 8771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:33.607295 8764 server_base.cc:1034] running on GCE node
I20260812 07:08:33.608047 8764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:33.608923 8764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:33.610160 8764 hybrid_clock.cc:648] HybridClock initialized: now 1786518513610132 us; error 42 us; skew 500 ppm
Aug 12 07:08:33 dist-test-slave-csg5 krb5kdc[8632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518513, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:33.614073 8764 init.cc:377] Logged in from keytab as oryx/127.31.156.65@KRBTEST.COM (short username oryx)
I20260812 07:08:33.615305 8764 webserver.cc:460] Webserver started at http://127.31.156.65:42507/ using document root <none> and password file <none>
I20260812 07:08:33.615965 8764 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:33.616074 8764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:33.616338 8764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:33.618736 8764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "8333b15d0d914b158fe4df290765b3c2"
format_stamp: "Formatted at 2026-08-12 07:08:33 on dist-test-slave-csg5"
server_key: "8177a05e522f1805c8462385011208a2"
server_key_iv: "0607fdc70a508c0d6b30a3f2dcaeab23"
server_key_version: "encryptionkey@0"
I20260812 07:08:33.619397 8764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "8333b15d0d914b158fe4df290765b3c2"
format_stamp: "Formatted at 2026-08-12 07:08:33 on dist-test-slave-csg5"
server_key: "8177a05e522f1805c8462385011208a2"
server_key_iv: "0607fdc70a508c0d6b30a3f2dcaeab23"
server_key_version: "encryptionkey@0"
I20260812 07:08:33.623653 8764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:08:33.626538 8784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:33.628399 8764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:08:33.628584 8764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "8333b15d0d914b158fe4df290765b3c2"
format_stamp: "Formatted at 2026-08-12 07:08:33 on dist-test-slave-csg5"
server_key: "8177a05e522f1805c8462385011208a2"
server_key_iv: "0607fdc70a508c0d6b30a3f2dcaeab23"
server_key_version: "encryptionkey@0"
I20260812 07:08:33.628749 8764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:33.648875 8764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:33.652882 8764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:33.653139 8764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:33.655145 8764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:33.655412 8764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:33.655547 8764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:33.655592 8764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:08:33 dist-test-slave-csg5 krb5kdc[8632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518513, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:33.667933 8764 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:39745
I20260812 07:08:33.669139 8764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:33.670049 8922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:39745 every 8 connection(s)
I20260812 07:08:33.671190 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:33.656553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:58639 (local address 127.31.156.126:37337)
0812 07:08:33.656696 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:33.656700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:33.657855 (+ 1155us) server_negotiation.cc:407] Connection header received
0812 07:08:33.659297 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:33.659305 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:33.659381 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:33.659494 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:33.660966 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.661529 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.662795 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.663116 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.663252 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:33.666575 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:33.666579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:33.666581 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:33.666612 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:33.668497 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:33.669103 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:33.669110 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:33.669114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:33.669225 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:33.669812 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:33.669815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:33.669817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:33.670095 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:33.670201 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:33.670928 (+ 727us) server_negotiation.cc:299] Negotiation successful
0812 07:08:33.671061 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:08:33.672083 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:33.657088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:37337 (local address 127.31.156.65:58639)
0812 07:08:33.657639 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:33.657696 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:33.658996 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:33.659720 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:33.659731 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:33.660193 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:33.660780 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:33.660797 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.661692 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.661695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:33.662614 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:33.662624 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.663311 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.663315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:33.663566 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:33.664377 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:33.664402 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:33.666354 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:33.668630 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:33.668635 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:33.668648 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:33.668984 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:33.669361 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:33.669365 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:33.669367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:33.669587 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:33.670308 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:33.670314 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:33.670600 (+ 286us) client_negotiation.cc:769] Sending connection context
0812 07:08:33.670854 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 07:08:33.671169 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":152,"threads_started":1}
I20260812 07:08:33.673934 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 8764
I20260812 07:08:33.673955 8924 heartbeater.cc:347] Connected to a master server at 127.31.156.126:37337
I20260812 07:08:33.674103 8924 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:33.674108 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:08:33.674422 32369 external_mini_cluster.cc:1428] Setting key ab5d8a747805322fe26c09af2b382288
I20260812 07:08:33.674853 8924 heartbeater.cc:511] Master 127.31.156.126:37337 requested a full tablet report, sending...
I20260812 07:08:33.676875 8681 ts_manager.cc:194] Registered new tserver with Master: 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745)
I20260812 07:08:33.678558 8681 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:58639
WARNING: no policy specified for oryx/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:08:33.687846 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:33.676786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:36747 (local address 127.31.156.126:37337)
0812 07:08:33.676922 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:33.676926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:33.679068 (+ 2142us) server_negotiation.cc:407] Connection header received
0812 07:08:33.679205 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:33.679210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:33.679275 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:33.679381 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:33.680310 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.680830 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.681864 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.682101 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.682425 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:33.683901 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:33.683905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:33.683908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:33.683953 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:33.685797 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:33.686343 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:33.686347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:33.686349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:33.686444 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:33.686833 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:33.686837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:33.686839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:33.687100 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:33.687227 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:33.687528 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 07:08:33.687656 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:08:33.687862 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:33.676744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:37337 (local address 127.31.156.65:36747)
0812 07:08:33.678949 (+ 2205us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:33.678967 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:33.679090 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:33.679429 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:33.679432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:33.679688 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:33.679814 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:33.679821 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.680983 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.680987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:33.681720 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:33.681732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.682420 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.682425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:33.682555 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:33.683282 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:33.683303 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:33.683722 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:33.685965 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:33.685969 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:33.685971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:33.686222 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:33.686558 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:33.686561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:33.686563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:33.686726 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:33.687214 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:33.687217 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:33.687337 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 07:08:33.687453 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 07:08:33.687656 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2078,"thread_start_us":125,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab.
Entry for principal oryx/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab.
I20260812 07:08:33.716693 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:37337
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab
--principal=oryx/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:33.843441 8933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:33.843778 8933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:33.843828 8933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:33.848102 8933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:33.848201 8933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:33.848249 8933 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:08:33.848285 8933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:33.854162 8933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab
--principal=oryx/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.156.126:37337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:33.855494 8933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:33.856559 8933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:33.867830 8940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:33.867874 8939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:33.867972 8933 server_base.cc:1034] running on GCE node
W20260812 07:08:33.868064 8942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:33.868657 8933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:33.869422 8933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:33.870715 8933 hybrid_clock.cc:648] HybridClock initialized: now 1786518513870632 us; error 35 us; skew 500 ppm
Aug 12 07:08:33 dist-test-slave-csg5 krb5kdc[8632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518513, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:33.873922 8933 init.cc:377] Logged in from keytab as oryx/127.31.156.66@KRBTEST.COM (short username oryx)
I20260812 07:08:33.875061 8933 webserver.cc:460] Webserver started at http://127.31.156.66:38947/ using document root <none> and password file <none>
I20260812 07:08:33.875659 8933 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:33.875766 8933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:33.876014 8933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:33.878296 8933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "21eb9cb925e5455fab6792b9008da016"
format_stamp: "Formatted at 2026-08-12 07:08:33 on dist-test-slave-csg5"
server_key: "d8f181a0276886144aa206b9cdd9e57c"
server_key_iv: "a751637ac028c081e56b7553d5f2512d"
server_key_version: "encryptionkey@0"
I20260812 07:08:33.878940 8933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "21eb9cb925e5455fab6792b9008da016"
format_stamp: "Formatted at 2026-08-12 07:08:33 on dist-test-slave-csg5"
server_key: "d8f181a0276886144aa206b9cdd9e57c"
server_key_iv: "a751637ac028c081e56b7553d5f2512d"
server_key_version: "encryptionkey@0"
I20260812 07:08:33.883358 8933 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:08:33.886399 8950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:33.888116 8933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:33.888352 8933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "21eb9cb925e5455fab6792b9008da016"
format_stamp: "Formatted at 2026-08-12 07:08:33 on dist-test-slave-csg5"
server_key: "d8f181a0276886144aa206b9cdd9e57c"
server_key_iv: "a751637ac028c081e56b7553d5f2512d"
server_key_version: "encryptionkey@0"
I20260812 07:08:33.888499 8933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:33.911007 8933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:33.915647 8933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:33.915920 8933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:33.918062 8933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:33.918166 8933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:33.918246 8933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:33.918485 8933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:08:33 dist-test-slave-csg5 krb5kdc[8632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518513, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:33.930882 8933 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:45283
I20260812 07:08:33.930920 9089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:45283 every 8 connection(s)
I20260812 07:08:33.932173 8933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:33.934082 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:33.918786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:50953 (local address 127.31.156.126:37337)
0812 07:08:33.918935 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:33.918940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:33.919972 (+ 1032us) server_negotiation.cc:407] Connection header received
0812 07:08:33.921051 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:33.921055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:33.921122 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:33.921234 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:33.922927 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.923552 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.924993 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.925355 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.925484 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:33.929200 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:33.929207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:33.929213 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:33.929256 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:33.931383 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:33.932029 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:33.932037 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:33.932041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:33.932153 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:33.932707 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:33.932711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:33.932713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:33.933038 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:33.933160 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:33.933721 (+ 561us) server_negotiation.cc:299] Negotiation successful
0812 07:08:33.933870 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 07:08:33.935030 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:33.919260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:37337 (local address 127.31.156.66:50953)
0812 07:08:33.919748 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:33.919816 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:33.920776 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:33.921434 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:33.921441 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:33.922047 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:33.922739 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:33.922764 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.923696 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.923699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:33.924862 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:33.924879 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.925539 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.925543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:33.925795 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:33.926583 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:33.926605 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:33.928951 (+ 2346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:33.931549 (+ 2598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:33.931555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:33.931569 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:33.931886 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:33.932331 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:33.932335 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:33.932337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:33.932555 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:33.933170 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:33.933177 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:33.933477 (+ 300us) client_negotiation.cc:769] Sending connection context
0812 07:08:33.933652 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 07:08:33.934019 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":141,"threads_started":1}
I20260812 07:08:33.936430 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 8933
I20260812 07:08:33.936586 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:08:33.936710 9090 heartbeater.cc:347] Connected to a master server at 127.31.156.126:37337
I20260812 07:08:33.936923 9090 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:33.936956 32369 external_mini_cluster.cc:1428] Setting key f2dbab8a0d42ac3e60882c93e7f3cf56
I20260812 07:08:33.937613 9090 heartbeater.cc:511] Master 127.31.156.126:37337 requested a full tablet report, sending...
I20260812 07:08:33.938750 8681 ts_manager.cc:194] Registered new tserver with Master: 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283)
I20260812 07:08:33.939517 8681 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:50953
I20260812 07:08:33.948524 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:33.939182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:37337 (local address 127.31.156.66:36145)
0812 07:08:33.939608 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:33.939626 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:33.939740 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:33.940016 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:33.940020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:33.940327 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:33.940457 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:33.940465 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.941431 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.941435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:33.942007 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:33.942015 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.942557 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.942561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:33.942710 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:33.943327 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:33.943340 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:33.943693 (+ 353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:33.946325 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:33.946329 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:33.946330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:33.946728 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:33.947151 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:33.947155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:33.947156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:33.947258 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:33.947930 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:33.947932 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:33.948036 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 07:08:33.948130 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 07:08:33.948350 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":145,"threads_started":1}
I20260812 07:08:33.948590 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:33.939249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:36145 (local address 127.31.156.126:37337)
0812 07:08:33.939365 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:33.939369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:33.939694 (+ 325us) server_negotiation.cc:407] Connection header received
0812 07:08:33.939837 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:33.939840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:33.939888 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:33.939990 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:33.940608 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.941260 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.942139 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:33.942398 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:33.942516 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:33.943858 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:33.943865 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:33.943871 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:33.943908 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:33.946145 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:33.946916 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:33.946923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:33.946925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:33.947035 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:33.947448 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:33.947453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:33.947457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:33.947821 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:33.947937 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:33.948231 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 07:08:33.948384 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for oryx/127.31.156.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab.
Entry for principal oryx/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab.
I20260812 07:08:33.976534 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:37337
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab
--principal=oryx/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:34.111095 9101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:34.111459 9101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:34.111546 9101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:34.115897 9101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:34.116024 9101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:34.116098 9101 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:08:34.116200 9101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:34.122062 9101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab
--principal=oryx/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.156.126:37337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:34.123409 9101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:34.124599 9101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:34.136668 9109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:34.136598 9107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:34.136649 9106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:34.137048 9101 server_base.cc:1034] running on GCE node
I20260812 07:08:34.137530 9101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:34.138316 9101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:34.139609 9101 hybrid_clock.cc:648] HybridClock initialized: now 1786518514139566 us; error 56 us; skew 500 ppm
Aug 12 07:08:34 dist-test-slave-csg5 krb5kdc[8632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518514, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:34.143042 9101 init.cc:377] Logged in from keytab as oryx/127.31.156.67@KRBTEST.COM (short username oryx)
I20260812 07:08:34.144121 9101 webserver.cc:460] Webserver started at http://127.31.156.67:45443/ using document root <none> and password file <none>
I20260812 07:08:34.144753 9101 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:34.144865 9101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:34.145136 9101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:34.147526 9101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "65cb8934dca145358c6f07283c2c91d8"
format_stamp: "Formatted at 2026-08-12 07:08:34 on dist-test-slave-csg5"
server_key: "5d77398a355244f67410ae8b99ed26b0"
server_key_iv: "adae7f9f4223ef98116012a6be058d66"
server_key_version: "encryptionkey@0"
I20260812 07:08:34.148192 9101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "65cb8934dca145358c6f07283c2c91d8"
format_stamp: "Formatted at 2026-08-12 07:08:34 on dist-test-slave-csg5"
server_key: "5d77398a355244f67410ae8b99ed26b0"
server_key_iv: "adae7f9f4223ef98116012a6be058d66"
server_key_version: "encryptionkey@0"
I20260812 07:08:34.152628 9101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 07:08:34.155830 9118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:34.157300 9101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:08:34.157490 9101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "65cb8934dca145358c6f07283c2c91d8"
format_stamp: "Formatted at 2026-08-12 07:08:34 on dist-test-slave-csg5"
server_key: "5d77398a355244f67410ae8b99ed26b0"
server_key_iv: "adae7f9f4223ef98116012a6be058d66"
server_key_version: "encryptionkey@0"
I20260812 07:08:34.157652 9101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:34.187816 9101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:34.193058 9101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:34.193307 9101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:34.195525 9101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:34.195636 9101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:34.195696 9101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:34.195719 9101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:08:34 dist-test-slave-csg5 krb5kdc[8632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518514, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:34.209517 9101 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:35671
I20260812 07:08:34.209964 9257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:35671 every 8 connection(s)
I20260812 07:08:34.210795 9101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:34.213443 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.196402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:49647 (local address 127.31.156.126:37337)
0812 07:08:34.196558 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.196563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.197700 (+ 1137us) server_negotiation.cc:407] Connection header received
0812 07:08:34.198857 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.198861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.198928 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.199021 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:34.201183 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.201929 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.203672 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.204120 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.204250 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.208366 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:34.208371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:34.208374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:34.208418 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:34.210646 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:34.211474 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:34.211478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:34.211480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:34.211603 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:34.212125 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:34.212129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:34.212130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:34.212358 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:34.212461 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.213134 (+ 673us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.213276 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:08:34.214402 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.196891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:37337 (local address 127.31.156.67:49647)
0812 07:08:34.197511 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.197560 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.198557 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.199384 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.199394 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.200032 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:34.200935 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.200965 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.202091 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.202095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.203507 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.203520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.204338 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.204343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.204557 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.205374 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:34.205403 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:34.208086 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:34.210871 (+ 2785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:34.210878 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:34.210898 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:34.211277 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:34.211795 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:34.211799 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:34.211801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:34.211988 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:34.212534 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:34.212540 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:34.212819 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.213031 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.213333 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":135,"threads_started":1}
I20260812 07:08:34.216091 9259 heartbeater.cc:347] Connected to a master server at 127.31.156.126:37337
I20260812 07:08:34.216467 9259 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:34.216961 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 9101
I20260812 07:08:34.217128 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:08:34.217171 9259 heartbeater.cc:511] Master 127.31.156.126:37337 requested a full tablet report, sending...
I20260812 07:08:34.217516 32369 external_mini_cluster.cc:1428] Setting key 775d13a01f786edc5e3a84a1b3c70c9a
I20260812 07:08:34.218518 8681 ts_manager.cc:194] Registered new tserver with Master: 65cb8934dca145358c6f07283c2c91d8 (127.31.156.67:35671)
I20260812 07:08:34.219359 8681 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:49647
I20260812 07:08:34.220355 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:08:34.229858 9265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.219356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:37337 (local address 127.31.156.67:37211)
0812 07:08:34.219805 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.219821 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.219937 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.220284 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.220287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.220633 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:34.220757 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.220779 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.221936 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.221939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.222819 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.222836 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.223396 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.223403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.223535 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.224163 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:34.224184 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:34.224668 (+ 484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:34.227603 (+ 2935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:34.227610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:34.227614 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:34.227970 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:34.228467 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:34.228470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:34.228471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:34.228574 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:34.229246 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:34.229248 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:34.229354 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.229452 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.229602 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":205,"threads_started":1}
I20260812 07:08:34.229984 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.219397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:37211 (local address 127.31.156.126:37337)
0812 07:08:34.219519 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.219523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.219915 (+ 392us) server_negotiation.cc:407] Connection header received
0812 07:08:34.220024 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.220027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.220077 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.220162 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:34.220934 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.221685 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.223011 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.223250 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.223353 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.224904 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:34.224907 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:34.224910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:34.224940 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:34.227419 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:34.228146 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:34.228150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:34.228152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:34.228292 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:34.228781 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:34.228787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:34.228792 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:34.229100 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:34.229231 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.229550 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.229724 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 07:08:34 dist-test-slave-csg5 krb5kdc[8632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518514, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:08:34 dist-test-slave-csg5 krb5kdc[8632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:34 dist-test-slave-csg5 krb5kdc[8632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:08:34.252777 8737 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:34.245202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.31.156.126:37337)
0812 07:08:34.245443 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.245447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.245463 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:08:34.245508 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.245510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.245553 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.245717 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:34.246546 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.247202 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.248147 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.248396 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.248516 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.252635 (+ 4119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58346: BlockingRecv error: recv got EOF from 127.0.0.1:58346 (error 108)
Metrics: {"server-negotiator.queue_time_us":143}
Aug 12 07:08:34 dist-test-slave-csg5 krb5kdc[8632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518514, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:34.266022 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.254824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.31.156.126:37337)
0812 07:08:34.254966 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.254970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.254987 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:08:34.255106 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.255108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.255152 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.255228 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:34.256460 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.257010 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.258019 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.258248 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.258417 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.262128 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:34.262132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:34.262134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:34.262173 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:34.264020 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:34.264594 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:34.264598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:34.264600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:34.264693 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:34.264998 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:34.265003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:34.265004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:34.265283 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:34.265432 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.265710 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.265848 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:08:34.269711 8680 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:08:34.273317 8680 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:34.291249 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.285183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.31.156.67:35671)
0812 07:08:34.285534 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.285549 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.285581 (+ 32us) server_negotiation.cc:407] Connection header received
0812 07:08:34.285685 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.285694 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.285935 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.286141 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:34.286465 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.287772 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.289507 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.290433 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.290558 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.290798 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.290956 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.291052 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":125,"threads_started":1}
I20260812 07:08:34.291265 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.284365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35671 (local address 127.0.0.1:56344)
0812 07:08:34.284831 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.284896 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.285028 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.286122 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.286129 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.286150 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:34.286319 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.286325 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.287936 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.287942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.289352 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.289365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.290577 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.290580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.290729 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.290780 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.290913 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.291059 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":22,"thread_start_us":120,"threads_started":1}
I20260812 07:08:34.292114 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.284556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.31.156.66:45283)
0812 07:08:34.285567 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.285582 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.285945 (+ 363us) server_negotiation.cc:407] Connection header received
0812 07:08:34.286161 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.286167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.286402 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.286531 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:34.286995 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.287844 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.289583 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.290433 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.290559 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.290798 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.291834 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.291960 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":884,"thread_start_us":122,"threads_started":1}
I20260812 07:08:34.292555 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.284366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:45283 (local address 127.0.0.1:42366)
0812 07:08:34.285838 (+ 1472us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.285853 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.286014 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.286758 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.286760 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.286782 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:34.286843 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.286847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.287988 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.287994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.289413 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.289426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.291523 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.291529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.291633 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.291676 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.292317 (+ 641us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.292416 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1414,"mutex_wait_us":1278,"thread_start_us":101,"threads_started":1}
I20260812 07:08:34.292730 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.285488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.31.156.65:39745)
0812 07:08:34.285885 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.285892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.285919 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:08:34.286103 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.286109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.286312 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.286496 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:34.286742 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.287822 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.290254 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.291095 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.291206 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.291358 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.292458 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.292581 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":93,"threads_started":1}
I20260812 07:08:34.292935 9290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.285057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:39745 (local address 127.0.0.1:36718)
0812 07:08:34.285838 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.285853 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.285952 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.286487 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.286490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.286506 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:34.286599 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.286604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.287952 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.287955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.290106 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.290116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.291243 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.291247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.291380 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.291693 (+ 313us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.291769 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.292741 (+ 972us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":203,"thread_start_us":132,"threads_started":1}
I20260812 07:08:34.295338 9181 tablet_service.cc:1467] Processing CreateTablet for tablet b22c435e163049129e733f84a695e0ae (DEFAULT_TABLE table=test-table [id=9031da48c0c647cfaba8b905770978d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:34.295481 9009 tablet_service.cc:1467] Processing CreateTablet for tablet b22c435e163049129e733f84a695e0ae (DEFAULT_TABLE table=test-table [id=9031da48c0c647cfaba8b905770978d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:34.295786 8841 tablet_service.cc:1467] Processing CreateTablet for tablet b22c435e163049129e733f84a695e0ae (DEFAULT_TABLE table=test-table [id=9031da48c0c647cfaba8b905770978d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:34.296655 9009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22c435e163049129e733f84a695e0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:34.296658 9181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22c435e163049129e733f84a695e0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:34.296754 8841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22c435e163049129e733f84a695e0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:34.304669 9292 tablet_bootstrap.cc:492] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: Bootstrap starting.
I20260812 07:08:34.306933 9294 tablet_bootstrap.cc:492] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: Bootstrap starting.
I20260812 07:08:34.307054 9293 tablet_bootstrap.cc:492] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: Bootstrap starting.
I20260812 07:08:34.309064 9292 tablet_bootstrap.cc:654] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:34.309885 9293 tablet_bootstrap.cc:654] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:34.310001 9294 tablet_bootstrap.cc:654] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:34.310259 9292 log.cc:824] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:34.310951 9293 log.cc:824] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:34.311208 9294 log.cc:824] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:34.313382 9293 tablet_bootstrap.cc:492] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: No bootstrap required, opened a new log
I20260812 07:08:34.313382 9292 tablet_bootstrap.cc:492] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: No bootstrap required, opened a new log
I20260812 07:08:34.313511 9294 tablet_bootstrap.cc:492] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: No bootstrap required, opened a new log
I20260812 07:08:34.313673 9293 ts_tablet_manager.cc:1397] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 07:08:34.313673 9292 ts_tablet_manager.cc:1397] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:08:34.313737 9294 ts_tablet_manager.cc:1397] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:08:34.318506 9294 raft_consensus.cc:348] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } }
I20260812 07:08:34.319070 9294 raft_consensus.cc:374] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:34.319147 9294 raft_consensus.cc:729] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65cb8934dca145358c6f07283c2c91d8, State: Initialized, Role: FOLLOWER
I20260812 07:08:34.319332 9292 raft_consensus.cc:348] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } }
I20260812 07:08:34.319654 9292 raft_consensus.cc:374] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:34.319720 9292 raft_consensus.cc:729] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21eb9cb925e5455fab6792b9008da016, State: Initialized, Role: FOLLOWER
I20260812 07:08:34.319571 9293 raft_consensus.cc:348] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } }
I20260812 07:08:34.319747 9294 consensus_queue.cc:260] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } }
I20260812 07:08:34.319861 9293 raft_consensus.cc:374] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:34.319931 9293 raft_consensus.cc:729] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8333b15d0d914b158fe4df290765b3c2, State: Initialized, Role: FOLLOWER
I20260812 07:08:34.320287 9292 consensus_queue.cc:260] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } }
I20260812 07:08:34.320463 9293 consensus_queue.cc:260] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } }
I20260812 07:08:34.320749 9259 heartbeater.cc:503] Master 127.31.156.126:37337 was elected leader, sending a full tablet report...
I20260812 07:08:34.321079 9294 ts_tablet_manager.cc:1428] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 07:08:34.321231 8924 heartbeater.cc:503] Master 127.31.156.126:37337 was elected leader, sending a full tablet report...
I20260812 07:08:34.321550 9293 ts_tablet_manager.cc:1428] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260812 07:08:34.322125 9090 heartbeater.cc:503] Master 127.31.156.126:37337 was elected leader, sending a full tablet report...
I20260812 07:08:34.322163 9292 ts_tablet_manager.cc:1428] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
W20260812 07:08:34.420953 8925 tablet.cc:2369] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:08:34.433771 9091 tablet.cc:2369] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:08:34.460496 9300 raft_consensus.cc:482] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:34.460757 9300 raft_consensus.cc:504] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } }
W20260812 07:08:34.462093 9260 tablet.cc:2369] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:08:34.462355 9300 leader_election.cc:288] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283), 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745)
I20260812 07:08:34.467478 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.463000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:49023 (local address 127.31.156.66:45283)
0812 07:08:34.463141 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.463146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.463164 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:08:34.463227 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.463231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.463289 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.463385 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:34.463821 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.464688 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.466276 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.466801 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.466946 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.467008 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.467203 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.467294 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:08:34.467551 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.462644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:45283 (local address 127.31.156.67:49023)
0812 07:08:34.462871 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.462890 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.463010 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.463485 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.463488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.463517 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:34.463663 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.463671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.464816 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.464820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.466126 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.466138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.466943 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.466947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.467067 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.467087 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.467184 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.467409 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":44}
I20260812 07:08:34.468365 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22c435e163049129e733f84a695e0ae" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eb9cb925e5455fab6792b9008da016" is_pre_election: true
I20260812 07:08:34.468654 9303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.462690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:39745 (local address 127.31.156.67:48457)
0812 07:08:34.463277 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.463292 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.463401 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.463711 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.463714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.463773 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:34.464550 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.464562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.465846 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.465850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.467137 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.467149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.468200 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.468204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.468301 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.468327 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.468419 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.468492 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":24,"thread_start_us":144,"threads_started":1}
I20260812 07:08:34.468775 9036 raft_consensus.cc:2454] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 0.
I20260812 07:08:34.468870 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.462935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:48457 (local address 127.31.156.65:39745)
0812 07:08:34.463104 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.463111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.463444 (+ 333us) server_negotiation.cc:407] Connection header received
0812 07:08:34.463506 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.463509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.463568 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.463663 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:34.464819 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.465671 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.467303 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.468029 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.468181 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.468245 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.468468 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.468664 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:08:34.469372 9120 leader_election.cc:302] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21eb9cb925e5455fab6792b9008da016, 65cb8934dca145358c6f07283c2c91d8; no voters:
I20260812 07:08:34.469568 8869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22c435e163049129e733f84a695e0ae" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8333b15d0d914b158fe4df290765b3c2" is_pre_election: true
I20260812 07:08:34.469703 9300 raft_consensus.cc:2781] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:34.469813 9300 raft_consensus.cc:482] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:34.469866 9300 raft_consensus.cc:3037] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:34.469920 8869 raft_consensus.cc:2454] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 0.
I20260812 07:08:34.471215 9300 raft_consensus.cc:504] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } }
I20260812 07:08:34.471680 9300 leader_election.cc:288] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 election: Requested vote from peers 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283), 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745)
I20260812 07:08:34.472268 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22c435e163049129e733f84a695e0ae" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eb9cb925e5455fab6792b9008da016"
I20260812 07:08:34.472400 8869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22c435e163049129e733f84a695e0ae" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8333b15d0d914b158fe4df290765b3c2"
I20260812 07:08:34.472396 9036 raft_consensus.cc:3037] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:34.472579 8869 raft_consensus.cc:3037] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:34.473683 9036 raft_consensus.cc:2454] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 1.
I20260812 07:08:34.473930 8869 raft_consensus.cc:2454] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 1.
I20260812 07:08:34.474112 9120 leader_election.cc:302] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21eb9cb925e5455fab6792b9008da016, 65cb8934dca145358c6f07283c2c91d8; no voters:
I20260812 07:08:34.474334 9300 raft_consensus.cc:2781] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:34.474632 9300 raft_consensus.cc:686] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 1 LEADER]: Becoming Leader. State: Replica: 65cb8934dca145358c6f07283c2c91d8, State: Running, Role: LEADER
I20260812 07:08:34.475057 9300 consensus_queue.cc:237] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } }
I20260812 07:08:34.478809 8678 catalog_manager.cc:5495] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65cb8934dca145358c6f07283c2c91d8 (127.31.156.67). New cstate: current_term: 1 leader_uuid: "65cb8934dca145358c6f07283c2c91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } health_report { overall_health: UNKNOWN } } }
I20260812 07:08:34.534828 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.530027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56356 (local address 127.31.156.67:35671)
0812 07:08:34.530161 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.530165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.530179 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:08:34.530273 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.530276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.530339 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.530440 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:34.530951 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.531554 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.532651 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.532931 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.533023 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.533443 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:34.534240 (+ 797us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:34.534402 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.534517 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.534615 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:08:34.542068 9036 raft_consensus.cc:1261] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Refusing update from remote peer 65cb8934dca145358c6f07283c2c91d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:34.542068 8869 raft_consensus.cc:1261] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Refusing update from remote peer 65cb8934dca145358c6f07283c2c91d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:34.542984 9304 consensus_queue.cc:1035] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:34.543202 9300 consensus_queue.cc:1035] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:34.551766 9307 mvcc.cc:205] Tried to move back new op lower bound from 7317579835555876864 to 7317579835293061120. Current Snapshot: MvccSnapshot[applied={T|T < 7317579835555876864}]
I20260812 07:08:34.555203 9308 mvcc.cc:205] Tried to move back new op lower bound from 7317579835555876864 to 7317579835293061120. Current Snapshot: MvccSnapshot[applied={T|T < 7317579835555876864 or (T in {7317579835555876864})}]
I20260812 07:08:34.562191 8678 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:34.562448 8678 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:34.566567 8678 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2: Sending DeleteTablet for 3 replicas of tablet b22c435e163049129e733f84a695e0ae
I20260812 07:08:34.567833 9181 tablet_service.cc:1514] Processing DeleteTablet for tablet b22c435e163049129e733f84a695e0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:34 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:56344
I20260812 07:08:34.567833 8841 tablet_service.cc:1514] Processing DeleteTablet for tablet b22c435e163049129e733f84a695e0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:34 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:36718
I20260812 07:08:34.568499 9316 tablet_replica.cc:330] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: stopping tablet replica
I20260812 07:08:34.568495 9009 tablet_service.cc:1514] Processing DeleteTablet for tablet b22c435e163049129e733f84a695e0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:34 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:42366
I20260812 07:08:34.568926 9316 raft_consensus.cc:2229] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:34.569073 9318 tablet_replica.cc:330] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: stopping tablet replica
I20260812 07:08:34.569325 9316 raft_consensus.cc:2258] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:34.569574 9318 raft_consensus.cc:2229] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:34.569849 9317 tablet_replica.cc:330] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: stopping tablet replica
I20260812 07:08:34.570075 9318 raft_consensus.cc:2258] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:34.570189 9317 raft_consensus.cc:2229] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:34.570372 8678 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 07:08:34.570590 9317 raft_consensus.cc:2258] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 07:08:34.570986 8678 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:34.572395 9318 ts_tablet_manager.cc:1905] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:34.572477 9317 ts_tablet_manager.cc:1905] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:34.577842 9317 ts_tablet_manager.cc:1918] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:34.578032 9317 log.cc:1196] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/b22c435e163049129e733f84a695e0ae
I20260812 07:08:34.578449 9317 ts_tablet_manager.cc:1939] T b22c435e163049129e733f84a695e0ae P 65cb8934dca145358c6f07283c2c91d8: Deleting consensus metadata
I20260812 07:08:34.578868 9318 ts_tablet_manager.cc:1918] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:34.579109 9318 log.cc:1196] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/b22c435e163049129e733f84a695e0ae
I20260812 07:08:34.580377 8664 catalog_manager.cc:4841] TS 65cb8934dca145358c6f07283c2c91d8 (127.31.156.67:35671): tablet b22c435e163049129e733f84a695e0ae (table test-table [id=9031da48c0c647cfaba8b905770978d9]) successfully deleted
I20260812 07:08:34.580492 9318 ts_tablet_manager.cc:1939] T b22c435e163049129e733f84a695e0ae P 21eb9cb925e5455fab6792b9008da016: Deleting consensus metadata
I20260812 07:08:34.582522 8841 tablet_service.cc:1467] Processing CreateTablet for tablet 858013ecc06d4ad992e7a1f1990c6c12 (DEFAULT_TABLE table=test-table [id=30e8531d17e14ba9bd52907c7096c4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:34.583016 8841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 858013ecc06d4ad992e7a1f1990c6c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:34.583055 9181 tablet_service.cc:1467] Processing CreateTablet for tablet 858013ecc06d4ad992e7a1f1990c6c12 (DEFAULT_TABLE table=test-table [id=30e8531d17e14ba9bd52907c7096c4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:34.583366 9181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 858013ecc06d4ad992e7a1f1990c6c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:34.586225 9294 tablet_bootstrap.cc:492] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8: Bootstrap starting.
I20260812 07:08:34.587352 9009 tablet_service.cc:1467] Processing CreateTablet for tablet 858013ecc06d4ad992e7a1f1990c6c12 (DEFAULT_TABLE table=test-table [id=30e8531d17e14ba9bd52907c7096c4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:34.587430 8665 catalog_manager.cc:4841] TS 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283): tablet b22c435e163049129e733f84a695e0ae (table test-table [id=9031da48c0c647cfaba8b905770978d9]) successfully deleted
I20260812 07:08:34.587735 9009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 858013ecc06d4ad992e7a1f1990c6c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:34.587893 9294 tablet_bootstrap.cc:654] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:34.589550 9316 ts_tablet_manager.cc:1905] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:34.591096 9293 tablet_bootstrap.cc:492] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2: Bootstrap starting.
I20260812 07:08:34.592289 9293 tablet_bootstrap.cc:654] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:34.593248 9316 ts_tablet_manager.cc:1918] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:34.593384 9316 log.cc:1196] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/wal/wals/b22c435e163049129e733f84a695e0ae
I20260812 07:08:34.593732 9316 ts_tablet_manager.cc:1939] T b22c435e163049129e733f84a695e0ae P 8333b15d0d914b158fe4df290765b3c2: Deleting consensus metadata
I20260812 07:08:34.593820 9292 tablet_bootstrap.cc:492] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016: Bootstrap starting.
I20260812 07:08:34.595404 9292 tablet_bootstrap.cc:654] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:34.595741 8667 catalog_manager.cc:4841] TS 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745): tablet b22c435e163049129e733f84a695e0ae (table test-table [id=9031da48c0c647cfaba8b905770978d9]) successfully deleted
I20260812 07:08:34.596503 9293 tablet_bootstrap.cc:492] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2: No bootstrap required, opened a new log
I20260812 07:08:34.596611 9293 ts_tablet_manager.cc:1397] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 07:08:34.597792 9293 raft_consensus.cc:348] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.597944 9293 raft_consensus.cc:374] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:34.597976 9293 raft_consensus.cc:729] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8333b15d0d914b158fe4df290765b3c2, State: Initialized, Role: FOLLOWER
I20260812 07:08:34.598116 9293 consensus_queue.cc:260] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.598496 9293 ts_tablet_manager.cc:1428] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:08:34.598753 9292 tablet_bootstrap.cc:492] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016: No bootstrap required, opened a new log
I20260812 07:08:34.598874 9292 ts_tablet_manager.cc:1397] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 07:08:34.600184 9294 tablet_bootstrap.cc:492] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8: No bootstrap required, opened a new log
I20260812 07:08:34.600306 9294 ts_tablet_manager.cc:1397] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260812 07:08:34.600790 9292 raft_consensus.cc:348] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.600888 9294 raft_consensus.cc:348] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.601028 9294 raft_consensus.cc:374] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:34.601075 9294 raft_consensus.cc:729] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65cb8934dca145358c6f07283c2c91d8, State: Initialized, Role: FOLLOWER
I20260812 07:08:34.601007 9292 raft_consensus.cc:374] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:34.601176 9292 raft_consensus.cc:729] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21eb9cb925e5455fab6792b9008da016, State: Initialized, Role: FOLLOWER
I20260812 07:08:34.601235 9294 consensus_queue.cc:260] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.601428 9292 consensus_queue.cc:260] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.601563 9294 ts_tablet_manager.cc:1428] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 07:08:34.601908 9292 ts_tablet_manager.cc:1428] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260812 07:08:34.734061 9301 raft_consensus.cc:482] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:34.734295 9301 raft_consensus.cc:504] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.735694 9301 leader_election.cc:288] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283), 65cb8934dca145358c6f07283c2c91d8 (127.31.156.67:35671)
I20260812 07:08:34.740556 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.736135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:34591 (local address 127.31.156.66:45283)
0812 07:08:34.736433 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.736438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.736757 (+ 319us) server_negotiation.cc:407] Connection header received
0812 07:08:34.736959 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.736962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.737042 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.737093 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:34.737489 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.738086 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.739377 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.739890 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.740028 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.740085 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.740313 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.740415 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:08:34.740571 9324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.736046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:45283 (local address 127.31.156.65:34591)
0812 07:08:34.736676 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.736690 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.736822 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.737219 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.737221 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.737235 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:34.737379 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.737384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.738207 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.738211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.739226 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.739239 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.740029 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.740033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.740161 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.740187 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.740275 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.740368 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":309,"thread_start_us":104,"threads_started":1}
I20260812 07:08:34.741178 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858013ecc06d4ad992e7a1f1990c6c12" candidate_uuid: "8333b15d0d914b158fe4df290765b3c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eb9cb925e5455fab6792b9008da016" is_pre_election: true
I20260812 07:08:34.741374 9036 raft_consensus.cc:2454] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8333b15d0d914b158fe4df290765b3c2 in term 0.
I20260812 07:08:34.741498 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.736259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:51933 (local address 127.31.156.67:35671)
0812 07:08:34.736382 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.736387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.736690 (+ 303us) server_negotiation.cc:407] Connection header received
0812 07:08:34.736874 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.736877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.736935 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.737028 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:08:34.737331 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.738128 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.739745 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.740332 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.740449 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.740499 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.741269 (+ 770us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.741365 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:08:34.741622 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.736046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35671 (local address 127.31.156.65:51933)
0812 07:08:34.736563 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.736580 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.736732 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.737057 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.737061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.737087 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:08:34.737215 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.737221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.738326 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.738332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.739588 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.739602 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.740566 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.740572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.741118 (+ 546us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.741139 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.741236 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.741413 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260812 07:08:34.742053 8786 leader_election.cc:302] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21eb9cb925e5455fab6792b9008da016, 8333b15d0d914b158fe4df290765b3c2; no voters:
I20260812 07:08:34.742210 9205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858013ecc06d4ad992e7a1f1990c6c12" candidate_uuid: "8333b15d0d914b158fe4df290765b3c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65cb8934dca145358c6f07283c2c91d8" is_pre_election: true
I20260812 07:08:34.742383 9301 raft_consensus.cc:2781] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:34.742445 9301 raft_consensus.cc:482] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:34.742473 9301 raft_consensus.cc:3037] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:34.742460 9205 raft_consensus.cc:2454] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8333b15d0d914b158fe4df290765b3c2 in term 0.
I20260812 07:08:34.744858 9301 raft_consensus.cc:504] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.745370 9301 leader_election.cc:288] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [CANDIDATE]: Term 1 election: Requested vote from peers 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283), 65cb8934dca145358c6f07283c2c91d8 (127.31.156.67:35671)
I20260812 07:08:34.745913 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858013ecc06d4ad992e7a1f1990c6c12" candidate_uuid: "8333b15d0d914b158fe4df290765b3c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eb9cb925e5455fab6792b9008da016"
I20260812 07:08:34.746107 9036 raft_consensus.cc:3037] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:34.746210 9205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858013ecc06d4ad992e7a1f1990c6c12" candidate_uuid: "8333b15d0d914b158fe4df290765b3c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65cb8934dca145358c6f07283c2c91d8"
I20260812 07:08:34.746358 9205 raft_consensus.cc:3037] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:34.747471 9205 raft_consensus.cc:2454] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8333b15d0d914b158fe4df290765b3c2 in term 1.
I20260812 07:08:34.747526 9036 raft_consensus.cc:2454] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8333b15d0d914b158fe4df290765b3c2 in term 1.
I20260812 07:08:34.747920 8785 leader_election.cc:302] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65cb8934dca145358c6f07283c2c91d8, 8333b15d0d914b158fe4df290765b3c2; no voters:
I20260812 07:08:34.748189 9301 raft_consensus.cc:2781] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:34.748492 9301 raft_consensus.cc:686] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 1 LEADER]: Becoming Leader. State: Replica: 8333b15d0d914b158fe4df290765b3c2, State: Running, Role: LEADER
I20260812 07:08:34.749234 9301 consensus_queue.cc:237] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.752204 8679 catalog_manager.cc:5495] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65). New cstate: current_term: 1 leader_uuid: "8333b15d0d914b158fe4df290765b3c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } health_report { overall_health: UNKNOWN } } }
Aug 12 07:08:34 dist-test-slave-csg5 krb5kdc[8632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518513, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.126@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:34.798326 9334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.784887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:37337 (local address 127.0.0.1:58362)
0812 07:08:34.785331 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.785347 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.785441 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.785896 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.785900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.787061 (+ 1161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:34.787209 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.787218 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.788495 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.788499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.789396 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.789410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.790045 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.790048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.790173 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.791058 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:34.791072 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:34.792927 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:34.795326 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:34.795340 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:34.795347 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:34.796048 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:34.796731 (+ 683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:34.796735 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:34.796737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:34.796847 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:34.797456 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:34.797464 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:34.797608 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.797977 (+ 369us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.798166 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":123,"threads_started":1}
I20260812 07:08:34.798521 9335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.784976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.31.156.126:37337)
0812 07:08:34.785360 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.785378 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.785415 (+ 37us) server_negotiation.cc:407] Connection header received
0812 07:08:34.785552 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.785554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.785600 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.785803 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:34.787467 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.788350 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.789552 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.789901 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.790015 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.793116 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:34.793120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:34.793122 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:34.793147 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:34.795092 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:34.796480 (+ 1388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:34.796484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:34.796485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:34.796586 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:34.796986 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:34.796989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:34.796991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:34.797259 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:34.797390 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.798137 (+ 747us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.798354 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":149,"threads_started":1}
I20260812 07:08:34.804080 8679 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:58362:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 07:08:34.804893 8679 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:34.811308 8841 tablet_service.cc:1467] Processing CreateTablet for tablet d99e511e442e475bb73711886cdcab52 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e616d6a42798449cbcadf603107847da]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:08:34.811308 9009 tablet_service.cc:1467] Processing CreateTablet for tablet d99e511e442e475bb73711886cdcab52 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e616d6a42798449cbcadf603107847da]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:08:34.811503 9181 tablet_service.cc:1467] Processing CreateTablet for tablet d99e511e442e475bb73711886cdcab52 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e616d6a42798449cbcadf603107847da]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:08:34.811836 9181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d99e511e442e475bb73711886cdcab52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:34.811836 8841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d99e511e442e475bb73711886cdcab52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:34.811837 9009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d99e511e442e475bb73711886cdcab52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:34.814853 9294 tablet_bootstrap.cc:492] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: Bootstrap starting.
I20260812 07:08:34.814939 9292 tablet_bootstrap.cc:492] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016: Bootstrap starting.
I20260812 07:08:34.814939 9293 tablet_bootstrap.cc:492] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2: Bootstrap starting.
I20260812 07:08:34.816092 9293 tablet_bootstrap.cc:654] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:34.816092 9292 tablet_bootstrap.cc:654] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:34.816138 9294 tablet_bootstrap.cc:654] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:34.817600 9293 tablet_bootstrap.cc:492] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2: No bootstrap required, opened a new log
I20260812 07:08:34.817600 9292 tablet_bootstrap.cc:492] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016: No bootstrap required, opened a new log
I20260812 07:08:34.817714 9292 ts_tablet_manager.cc:1397] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:34.817714 9293 ts_tablet_manager.cc:1397] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:34.818245 9293 raft_consensus.cc:348] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.818234 9292 raft_consensus.cc:348] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.818631 9292 raft_consensus.cc:374] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:34.818773 9293 raft_consensus.cc:374] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:34.818804 9292 raft_consensus.cc:729] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21eb9cb925e5455fab6792b9008da016, State: Initialized, Role: FOLLOWER
I20260812 07:08:34.818835 9293 raft_consensus.cc:729] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8333b15d0d914b158fe4df290765b3c2, State: Initialized, Role: FOLLOWER
I20260812 07:08:34.819003 9293 consensus_queue.cc:260] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.819003 9292 consensus_queue.cc:260] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.819372 9293 ts_tablet_manager.cc:1428] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:08:34.819358 9292 ts_tablet_manager.cc:1428] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 07:08:34.819679 9301 tablet_replica.cc:439] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } }
I20260812 07:08:34.819693 9302 tablet_replica.cc:439] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } }
I20260812 07:08:34.819857 9302 tablet_replica.cc:442] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:34.819857 9301 tablet_replica.cc:442] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:34.819973 9294 tablet_bootstrap.cc:492] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: No bootstrap required, opened a new log
I20260812 07:08:34.820097 9294 ts_tablet_manager.cc:1397] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 07:08:34.820621 9294 raft_consensus.cc:348] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.821141 9294 raft_consensus.cc:374] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:34.821208 9294 raft_consensus.cc:729] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65cb8934dca145358c6f07283c2c91d8, State: Initialized, Role: FOLLOWER
I20260812 07:08:34.821388 9294 consensus_queue.cc:260] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.821846 9294 ts_tablet_manager.cc:1428] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 07:08:34.821794 9304 tablet_replica.cc:439] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } }
I20260812 07:08:34.821934 9304 tablet_replica.cc:442] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:34.843772 9304 raft_consensus.cc:482] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:34.843974 9304 raft_consensus.cc:504] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.844533 9304 leader_election.cc:288] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745), 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283)
I20260812 07:08:34.845094 8869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d99e511e442e475bb73711886cdcab52" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8333b15d0d914b158fe4df290765b3c2" is_pre_election: true
I20260812 07:08:34.845150 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d99e511e442e475bb73711886cdcab52" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eb9cb925e5455fab6792b9008da016" is_pre_election: true
I20260812 07:08:34.845333 8869 raft_consensus.cc:2454] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 0.
I20260812 07:08:34.845368 9036 raft_consensus.cc:2454] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 0.
I20260812 07:08:34.845790 9120 leader_election.cc:302] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21eb9cb925e5455fab6792b9008da016, 65cb8934dca145358c6f07283c2c91d8; no voters:
I20260812 07:08:34.846035 9304 raft_consensus.cc:2781] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:34.846107 9304 raft_consensus.cc:482] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:34.846150 9304 raft_consensus.cc:3037] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:34.847980 9304 raft_consensus.cc:504] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.848773 9304 leader_election.cc:288] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 election: Requested vote from peers 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745), 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283)
I20260812 07:08:34.849309 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d99e511e442e475bb73711886cdcab52" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eb9cb925e5455fab6792b9008da016"
I20260812 07:08:34.849445 9036 raft_consensus.cc:3037] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:34.849504 8869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d99e511e442e475bb73711886cdcab52" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8333b15d0d914b158fe4df290765b3c2"
I20260812 07:08:34.849987 8869 raft_consensus.cc:3037] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:34.851264 8869 raft_consensus.cc:2454] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 1.
I20260812 07:08:34.851292 9036 raft_consensus.cc:2454] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 1.
I20260812 07:08:34.851729 9123 leader_election.cc:302] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 21eb9cb925e5455fab6792b9008da016, 65cb8934dca145358c6f07283c2c91d8, 8333b15d0d914b158fe4df290765b3c2; no voters:
I20260812 07:08:34.851959 9304 raft_consensus.cc:2781] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:34.852075 9304 raft_consensus.cc:686] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [term 1 LEADER]: Becoming Leader. State: Replica: 65cb8934dca145358c6f07283c2c91d8, State: Running, Role: LEADER
I20260812 07:08:34.852239 9304 consensus_queue.cc:237] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:34.853365 9300 tablet_replica.cc:439] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: TxnStatusTablet state changed. Reason: New leader 65cb8934dca145358c6f07283c2c91d8. Latest consensus state: current_term: 1 leader_uuid: "65cb8934dca145358c6f07283c2c91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } }
I20260812 07:08:34.853494 9300 tablet_replica.cc:442] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:08:34.853791 9340 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 07:08:34.853982 9340 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 07:08:34.854382 8679 catalog_manager.cc:5495] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65cb8934dca145358c6f07283c2c91d8 (127.31.156.67). New cstate: current_term: 1 leader_uuid: "65cb8934dca145358c6f07283c2c91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } health_report { overall_health: HEALTHY } } }
I20260812 07:08:34.874414 9334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.870405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35671 (local address 127.0.0.1:56358)
0812 07:08:34.870578 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.870597 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.870754 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.871063 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.871066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.871074 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:08:34.871189 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.871194 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.871989 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.871993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.872711 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.872725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.873458 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.873461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.873594 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.873671 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:08:34.874052 (+ 381us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:08:34.874069 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.874199 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.874291 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:08:34.874418 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.870588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.31.156.67:35671)
0812 07:08:34.870767 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.870772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.870799 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:08:34.870881 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.870886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.870944 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.871028 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:34.871329 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.871861 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.872864 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.873272 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.873400 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.873842 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:34.873946 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:34.874029 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.874172 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.874245 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 07:08:34.878037 9036 raft_consensus.cc:1261] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Refusing update from remote peer 65cb8934dca145358c6f07283c2c91d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:34.878043 8869 raft_consensus.cc:1261] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Refusing update from remote peer 65cb8934dca145358c6f07283c2c91d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:34.878611 9300 consensus_queue.cc:1035] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:34.878901 9304 consensus_queue.cc:1035] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:34.880689 9301 tablet_replica.cc:439] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2: TxnStatusTablet state changed. Reason: New leader 65cb8934dca145358c6f07283c2c91d8. Latest consensus state: current_term: 1 leader_uuid: "65cb8934dca145358c6f07283c2c91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } }
I20260812 07:08:34.880829 9301 tablet_replica.cc:442] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:34.880744 9302 tablet_replica.cc:439] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016: TxnStatusTablet state changed. Reason: New leader 65cb8934dca145358c6f07283c2c91d8. Latest consensus state: current_term: 1 leader_uuid: "65cb8934dca145358c6f07283c2c91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } }
I20260812 07:08:34.880878 9302 tablet_replica.cc:442] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:34.882975 9304 tablet_replica.cc:439] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65cb8934dca145358c6f07283c2c91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } }
I20260812 07:08:34.883131 9304 tablet_replica.cc:442] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:08:34.883908 9302 tablet_replica.cc:439] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65cb8934dca145358c6f07283c2c91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } }
I20260812 07:08:34.884033 9302 tablet_replica.cc:442] T d99e511e442e475bb73711886cdcab52 P 21eb9cb925e5455fab6792b9008da016: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:34.888819 9304 tablet_replica.cc:439] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65cb8934dca145358c6f07283c2c91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } }
I20260812 07:08:34.888990 9304 tablet_replica.cc:442] T d99e511e442e475bb73711886cdcab52 P 65cb8934dca145358c6f07283c2c91d8: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:08:34.890779 9301 tablet_replica.cc:439] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65cb8934dca145358c6f07283c2c91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } } }
I20260812 07:08:34.890966 9301 tablet_replica.cc:442] T d99e511e442e475bb73711886cdcab52 P 8333b15d0d914b158fe4df290765b3c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:08:34.895849 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.890586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36728 (local address 127.31.156.65:39745)
0812 07:08:34.890726 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.890733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.890749 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:08:34.890854 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.890857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.890934 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.891064 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:34.891441 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.892375 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.893601 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.894007 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.894131 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.894612 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:34.895342 (+ 730us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:34.895477 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.895579 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.895653 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 07:08:34.911196 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.906009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:50405 (local address 127.31.156.67:35671)
0812 07:08:34.906207 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.906212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.906673 (+ 461us) server_negotiation.cc:407] Connection header received
0812 07:08:34.906746 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.906749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.906810 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.906909 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:34.907283 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.908069 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.909381 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.909731 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.909870 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.910437 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:34.910558 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:34.910678 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.910943 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.911020 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 07:08:34.911317 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.905841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35671 (local address 127.31.156.65:50405)
0812 07:08:34.906290 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.906309 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.906453 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.906962 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.906965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.906990 (+ 25us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:08:34.907115 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.907122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.908315 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.908318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.909241 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.909255 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.909971 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.909976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.910075 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.910240 (+ 165us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:08:34.910805 (+ 565us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:08:34.910817 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.910912 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.911087 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":140,"threads_started":1}
I20260812 07:08:34.920481 9036 raft_consensus.cc:1261] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Refusing update from remote peer 8333b15d0d914b158fe4df290765b3c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:34.920699 9205 raft_consensus.cc:1261] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Refusing update from remote peer 8333b15d0d914b158fe4df290765b3c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:34.921164 9325 consensus_queue.cc:1035] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:34.921742 9301 consensus_queue.cc:1035] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:34.960805 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.955962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:49359 (local address 127.31.156.65:39745)
0812 07:08:34.956094 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:34.956099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:34.956669 (+ 570us) server_negotiation.cc:407] Connection header received
0812 07:08:34.956788 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:34.956792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:34.956852 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:34.956938 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:34.957221 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.958033 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.959356 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.959725 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.959865 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.960326 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:34.960424 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:34.960497 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:34.960618 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 07:08:34.960679 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:08:34.960868 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:34.955846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:39745 (local address 127.31.156.67:49359)
0812 07:08:34.956560 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:34.956576 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:34.956691 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:34.956968 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:34.956971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:34.956980 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:08:34.957100 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.957107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.958195 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.958200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.959193 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:34.959205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:34.959950 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:34.959954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:34.960066 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:34.960131 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:08:34.960523 (+ 392us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:08:34.960530 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:08:34.960604 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 07:08:34.960700 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":133,"threads_started":1}
I20260812 07:08:34.992772 8678 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:34.992972 8678 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:34.995534 8678 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2: Sending DeleteTablet for 3 replicas of tablet 858013ecc06d4ad992e7a1f1990c6c12
I20260812 07:08:34.996369 9181 tablet_service.cc:1514] Processing DeleteTablet for tablet 858013ecc06d4ad992e7a1f1990c6c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:34 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:56344
I20260812 07:08:34.996444 9009 tablet_service.cc:1514] Processing DeleteTablet for tablet 858013ecc06d4ad992e7a1f1990c6c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:34 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:42366
I20260812 07:08:34.996595 9317 tablet_replica.cc:330] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8: stopping tablet replica
I20260812 07:08:34.996652 9318 tablet_replica.cc:330] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016: stopping tablet replica
I20260812 07:08:34.996729 9317 raft_consensus.cc:2229] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:34.996768 9318 raft_consensus.cc:2229] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:34.996878 9317 raft_consensus.cc:2258] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:34.996917 9318 raft_consensus.cc:2258] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:34.997545 8841 tablet_service.cc:1514] Processing DeleteTablet for tablet 858013ecc06d4ad992e7a1f1990c6c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:34 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:36718
I20260812 07:08:34.997761 9316 tablet_replica.cc:330] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2: stopping tablet replica
I20260812 07:08:34.997874 9316 raft_consensus.cc:2229] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:34.998091 9317 ts_tablet_manager.cc:1905] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:34.998224 9316 raft_consensus.cc:2258] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:34.999138 9318 ts_tablet_manager.cc:1905] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:35.001355 9317 ts_tablet_manager.cc:1918] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:08:35.001441 9317 log.cc:1196] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/858013ecc06d4ad992e7a1f1990c6c12
I20260812 07:08:35.001711 9317 ts_tablet_manager.cc:1939] T 858013ecc06d4ad992e7a1f1990c6c12 P 65cb8934dca145358c6f07283c2c91d8: Deleting consensus metadata
I20260812 07:08:35.002307 9318 ts_tablet_manager.cc:1918] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:08:35.002394 9318 log.cc:1196] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/858013ecc06d4ad992e7a1f1990c6c12
I20260812 07:08:35.002782 9318 ts_tablet_manager.cc:1939] T 858013ecc06d4ad992e7a1f1990c6c12 P 21eb9cb925e5455fab6792b9008da016: Deleting consensus metadata
I20260812 07:08:35.002785 9316 ts_tablet_manager.cc:1905] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:35.003741 8665 catalog_manager.cc:4841] TS 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283): tablet 858013ecc06d4ad992e7a1f1990c6c12 (table test-table [id=30e8531d17e14ba9bd52907c7096c4cc]) successfully deleted
I20260812 07:08:35.005748 9316 ts_tablet_manager.cc:1918] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:08:35.005841 9316 log.cc:1196] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/wal/wals/858013ecc06d4ad992e7a1f1990c6c12
I20260812 07:08:35.006093 9316 ts_tablet_manager.cc:1939] T 858013ecc06d4ad992e7a1f1990c6c12 P 8333b15d0d914b158fe4df290765b3c2: Deleting consensus metadata
I20260812 07:08:35.010799 8667 catalog_manager.cc:4841] TS 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745): tablet 858013ecc06d4ad992e7a1f1990c6c12 (table test-table [id=30e8531d17e14ba9bd52907c7096c4cc]) successfully deleted
I20260812 07:08:35.011180 8664 catalog_manager.cc:4841] TS 65cb8934dca145358c6f07283c2c91d8 (127.31.156.67:35671): tablet 858013ecc06d4ad992e7a1f1990c6c12 (table test-table [id=30e8531d17e14ba9bd52907c7096c4cc]) successfully deleted
Aug 12 07:08:35 dist-test-slave-csg5 krb5kdc[8632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:08:35 dist-test-slave-csg5 krb5kdc[8632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:35 dist-test-slave-csg5 krb5kdc[8632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:08:35.047545 9335 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:35.038524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58372 (local address 127.31.156.126:37337)
0812 07:08:35.038662 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:35.038667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:35.038927 (+ 260us) server_negotiation.cc:407] Connection header received
0812 07:08:35.039067 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:35.039071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:35.039133 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:35.039225 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:35.040220 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:35.041049 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:35.042411 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:35.042859 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:35.043004 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:35.047383 (+ 4379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58372: BlockingRecv error: recv got EOF from 127.0.0.1:58372 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 07:08:35 dist-test-slave-csg5 krb5kdc[8632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM
I20260812 07:08:35.061564 9335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:35.049942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58384 (local address 127.31.156.126:37337)
0812 07:08:35.050077 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:35.050082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:35.050611 (+ 529us) server_negotiation.cc:407] Connection header received
0812 07:08:35.050802 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:35.050805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:35.050869 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:35.050966 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:35.051828 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:35.052719 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:35.053949 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:35.054284 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:35.054399 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:35.057608 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:35.057613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:35.057616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:35.057660 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:35.059462 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:35.060012 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:35.060016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:35.060018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:35.060119 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:35.060459 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:35.060463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:35.060465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:35.060690 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:35.060866 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:35.061147 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 07:08:35.061366 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:08:35.064985 8678 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:08:35.065579 8678 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:35.071262 8841 tablet_service.cc:1467] Processing CreateTablet for tablet e80f9a7ac1a04c0a92664e7c3d96d5c8 (DEFAULT_TABLE table=test-table [id=5902c86243df4ac6a7c66f565bbc6198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:35.071259 9009 tablet_service.cc:1467] Processing CreateTablet for tablet e80f9a7ac1a04c0a92664e7c3d96d5c8 (DEFAULT_TABLE table=test-table [id=5902c86243df4ac6a7c66f565bbc6198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:35.071628 8841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e80f9a7ac1a04c0a92664e7c3d96d5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:35.071636 9009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e80f9a7ac1a04c0a92664e7c3d96d5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:35.073837 9292 tablet_bootstrap.cc:492] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016: Bootstrap starting.
I20260812 07:08:35.074156 9293 tablet_bootstrap.cc:492] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2: Bootstrap starting.
I20260812 07:08:35.074889 9292 tablet_bootstrap.cc:654] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:35.075202 9293 tablet_bootstrap.cc:654] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:35.076229 9181 tablet_service.cc:1467] Processing CreateTablet for tablet e80f9a7ac1a04c0a92664e7c3d96d5c8 (DEFAULT_TABLE table=test-table [id=5902c86243df4ac6a7c66f565bbc6198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:35.076524 9181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e80f9a7ac1a04c0a92664e7c3d96d5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:35.076642 9293 tablet_bootstrap.cc:492] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2: No bootstrap required, opened a new log
I20260812 07:08:35.076820 9293 ts_tablet_manager.cc:1397] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 07:08:35.077280 9293 raft_consensus.cc:348] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.077379 9293 raft_consensus.cc:374] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:35.077401 9293 raft_consensus.cc:729] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8333b15d0d914b158fe4df290765b3c2, State: Initialized, Role: FOLLOWER
I20260812 07:08:35.077546 9293 consensus_queue.cc:260] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.077838 9293 ts_tablet_manager.cc:1428] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 07:08:35.079178 9294 tablet_bootstrap.cc:492] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8: Bootstrap starting.
I20260812 07:08:35.080065 9294 tablet_bootstrap.cc:654] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:35.080964 9292 tablet_bootstrap.cc:492] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016: No bootstrap required, opened a new log
I20260812 07:08:35.081079 9292 ts_tablet_manager.cc:1397] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20260812 07:08:35.081724 9294 tablet_bootstrap.cc:492] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8: No bootstrap required, opened a new log
I20260812 07:08:35.081647 9292 raft_consensus.cc:348] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.081789 9292 raft_consensus.cc:374] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:35.081811 9294 ts_tablet_manager.cc:1397] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 07:08:35.081828 9292 raft_consensus.cc:729] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21eb9cb925e5455fab6792b9008da016, State: Initialized, Role: FOLLOWER
I20260812 07:08:35.081975 9292 consensus_queue.cc:260] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.082217 9294 raft_consensus.cc:348] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.082297 9292 ts_tablet_manager.cc:1428] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 07:08:35.082327 9294 raft_consensus.cc:374] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:35.082358 9294 raft_consensus.cc:729] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65cb8934dca145358c6f07283c2c91d8, State: Initialized, Role: FOLLOWER
I20260812 07:08:35.082473 9294 consensus_queue.cc:260] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.082788 9294 ts_tablet_manager.cc:1428] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:08:35.519232 9304 raft_consensus.cc:482] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:35.519496 9304 raft_consensus.cc:504] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.519977 9304 leader_election.cc:288] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745), 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283)
I20260812 07:08:35.520547 8869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80f9a7ac1a04c0a92664e7c3d96d5c8" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8333b15d0d914b158fe4df290765b3c2" is_pre_election: true
I20260812 07:08:35.520567 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80f9a7ac1a04c0a92664e7c3d96d5c8" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eb9cb925e5455fab6792b9008da016" is_pre_election: true
I20260812 07:08:35.520711 9036 raft_consensus.cc:2454] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 0.
I20260812 07:08:35.520728 8869 raft_consensus.cc:2454] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 0.
I20260812 07:08:35.521092 9120 leader_election.cc:302] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21eb9cb925e5455fab6792b9008da016, 65cb8934dca145358c6f07283c2c91d8; no voters:
I20260812 07:08:35.521413 9304 raft_consensus.cc:2781] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:35.521486 9304 raft_consensus.cc:482] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:35.521539 9304 raft_consensus.cc:3037] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:35.522866 9304 raft_consensus.cc:504] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.523320 9304 leader_election.cc:288] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 election: Requested vote from peers 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745), 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283)
I20260812 07:08:35.523811 8869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80f9a7ac1a04c0a92664e7c3d96d5c8" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8333b15d0d914b158fe4df290765b3c2"
I20260812 07:08:35.524024 8869 raft_consensus.cc:3037] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:35.523955 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80f9a7ac1a04c0a92664e7c3d96d5c8" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eb9cb925e5455fab6792b9008da016"
I20260812 07:08:35.524276 9036 raft_consensus.cc:3037] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:35.525308 9036 raft_consensus.cc:2454] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 1.
I20260812 07:08:35.525314 8869 raft_consensus.cc:2454] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 1.
I20260812 07:08:35.525717 9120 leader_election.cc:302] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21eb9cb925e5455fab6792b9008da016, 65cb8934dca145358c6f07283c2c91d8; no voters:
I20260812 07:08:35.525947 9304 raft_consensus.cc:2781] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:35.526132 9304 raft_consensus.cc:686] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 1 LEADER]: Becoming Leader. State: Replica: 65cb8934dca145358c6f07283c2c91d8, State: Running, Role: LEADER
I20260812 07:08:35.526350 9304 consensus_queue.cc:237] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.528604 8678 catalog_manager.cc:5495] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65cb8934dca145358c6f07283c2c91d8 (127.31.156.67). New cstate: current_term: 1 leader_uuid: "65cb8934dca145358c6f07283c2c91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } health_report { overall_health: HEALTHY } } }
I20260812 07:08:35.539673 9379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:35.535995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.31.156.67:35671)
0812 07:08:35.536225 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:35.536229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:35.536242 (+ 13us) server_negotiation.cc:407] Connection header received
0812 07:08:35.536368 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:35.536371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:35.536421 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:35.536498 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:35.536764 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:35.537325 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:35.538276 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:35.538571 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:35.538726 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:35.539050 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:35.539140 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:35.539199 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:35.539468 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 07:08:35.539532 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
I20260812 07:08:35.542256 9036 raft_consensus.cc:1261] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Refusing update from remote peer 65cb8934dca145358c6f07283c2c91d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:35.542263 8869 raft_consensus.cc:1261] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Refusing update from remote peer 65cb8934dca145358c6f07283c2c91d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:35.542968 9376 consensus_queue.cc:1035] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:35.543280 9304 consensus_queue.cc:1035] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:35.549912 9382 mvcc.cc:205] Tried to move back new op lower bound from 7317579839655985152 to 7317579839597998080. Current Snapshot: MvccSnapshot[applied={T|T < 7317579839655985152 or (T in {7317579839655985152})}]
I20260812 07:08:35.552191 8679 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:35.552392 8679 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:35.554605 8679 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2: Sending DeleteTablet for 3 replicas of tablet e80f9a7ac1a04c0a92664e7c3d96d5c8
I20260812 07:08:35.555490 9009 tablet_service.cc:1514] Processing DeleteTablet for tablet e80f9a7ac1a04c0a92664e7c3d96d5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:35 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:42366
I20260812 07:08:35.555501 9181 tablet_service.cc:1514] Processing DeleteTablet for tablet e80f9a7ac1a04c0a92664e7c3d96d5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:35 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:56344
I20260812 07:08:35.555552 8841 tablet_service.cc:1514] Processing DeleteTablet for tablet e80f9a7ac1a04c0a92664e7c3d96d5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:35 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:36718
I20260812 07:08:35.555902 9391 tablet_replica.cc:330] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016: stopping tablet replica
I20260812 07:08:35.555966 9393 tablet_replica.cc:330] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8: stopping tablet replica
I20260812 07:08:35.556064 9391 raft_consensus.cc:2229] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:35.556129 9393 raft_consensus.cc:2229] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:35.556223 9391 raft_consensus.cc:2258] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:35.556371 9393 raft_consensus.cc:2258] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:35.556479 9392 tablet_replica.cc:330] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2: stopping tablet replica
I20260812 07:08:35.556631 9392 raft_consensus.cc:2229] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:35.556527 8679 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 07:08:35.556800 9392 raft_consensus.cc:2258] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 07:08:35.557014 8679 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:08:35.557165 9393 ts_tablet_manager.cc:1905] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:35.557507 9391 ts_tablet_manager.cc:1905] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:35.558203 9392 ts_tablet_manager.cc:1905] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:35.560190 9393 ts_tablet_manager.cc:1918] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:35.560281 9393 log.cc:1196] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/e80f9a7ac1a04c0a92664e7c3d96d5c8
I20260812 07:08:35.560629 9393 ts_tablet_manager.cc:1939] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 65cb8934dca145358c6f07283c2c91d8: Deleting consensus metadata
I20260812 07:08:35.560701 9391 ts_tablet_manager.cc:1918] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:35.560853 9391 log.cc:1196] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/e80f9a7ac1a04c0a92664e7c3d96d5c8
I20260812 07:08:35.561122 9391 ts_tablet_manager.cc:1939] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 21eb9cb925e5455fab6792b9008da016: Deleting consensus metadata
I20260812 07:08:35.561857 9392 ts_tablet_manager.cc:1918] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:08:35.561954 9392 log.cc:1196] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-0/wal/wals/e80f9a7ac1a04c0a92664e7c3d96d5c8
I20260812 07:08:35.562140 8665 catalog_manager.cc:4841] TS 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283): tablet e80f9a7ac1a04c0a92664e7c3d96d5c8 (table test-table [id=5902c86243df4ac6a7c66f565bbc6198]) successfully deleted
I20260812 07:08:35.562143 8664 catalog_manager.cc:4841] TS 65cb8934dca145358c6f07283c2c91d8 (127.31.156.67:35671): tablet e80f9a7ac1a04c0a92664e7c3d96d5c8 (table test-table [id=5902c86243df4ac6a7c66f565bbc6198]) successfully deleted
I20260812 07:08:35.562320 9392 ts_tablet_manager.cc:1939] T e80f9a7ac1a04c0a92664e7c3d96d5c8 P 8333b15d0d914b158fe4df290765b3c2: Deleting consensus metadata
I20260812 07:08:35.563108 8667 catalog_manager.cc:4841] TS 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745): tablet e80f9a7ac1a04c0a92664e7c3d96d5c8 (table test-table [id=5902c86243df4ac6a7c66f565bbc6198]) successfully deleted
I20260812 07:08:35.564692 8841 tablet_service.cc:1467] Processing CreateTablet for tablet 5a03ca6978a84476bced3ffdb665796d (DEFAULT_TABLE table=test-table [id=46ab38c360284d348baf4b7732038e49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:35.564939 9009 tablet_service.cc:1467] Processing CreateTablet for tablet 5a03ca6978a84476bced3ffdb665796d (DEFAULT_TABLE table=test-table [id=46ab38c360284d348baf4b7732038e49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:35.565048 8841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a03ca6978a84476bced3ffdb665796d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:35.565120 9181 tablet_service.cc:1467] Processing CreateTablet for tablet 5a03ca6978a84476bced3ffdb665796d (DEFAULT_TABLE table=test-table [id=46ab38c360284d348baf4b7732038e49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:08:35.565375 9009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a03ca6978a84476bced3ffdb665796d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:35.565537 9181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a03ca6978a84476bced3ffdb665796d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:35.568231 9293 tablet_bootstrap.cc:492] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2: Bootstrap starting.
I20260812 07:08:35.568598 9294 tablet_bootstrap.cc:492] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8: Bootstrap starting.
I20260812 07:08:35.569305 9293 tablet_bootstrap.cc:654] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:35.569469 9294 tablet_bootstrap.cc:654] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:35.570050 9292 tablet_bootstrap.cc:492] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016: Bootstrap starting.
I20260812 07:08:35.570951 9293 tablet_bootstrap.cc:492] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2: No bootstrap required, opened a new log
I20260812 07:08:35.571040 9293 ts_tablet_manager.cc:1397] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 07:08:35.571086 9294 tablet_bootstrap.cc:492] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8: No bootstrap required, opened a new log
I20260812 07:08:35.571163 9294 ts_tablet_manager.cc:1397] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:35.571306 9292 tablet_bootstrap.cc:654] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:35.571512 9294 raft_consensus.cc:348] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.571606 9294 raft_consensus.cc:374] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:35.571627 9294 raft_consensus.cc:729] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65cb8934dca145358c6f07283c2c91d8, State: Initialized, Role: FOLLOWER
I20260812 07:08:35.571561 9293 raft_consensus.cc:348] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.571693 9293 raft_consensus.cc:374] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:35.571727 9293 raft_consensus.cc:729] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8333b15d0d914b158fe4df290765b3c2, State: Initialized, Role: FOLLOWER
I20260812 07:08:35.571786 9294 consensus_queue.cc:260] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.571894 9293 consensus_queue.cc:260] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.572109 9294 ts_tablet_manager.cc:1428] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:08:35.572227 9293 ts_tablet_manager.cc:1428] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:08:35.572692 9292 tablet_bootstrap.cc:492] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016: No bootstrap required, opened a new log
I20260812 07:08:35.572790 9292 ts_tablet_manager.cc:1397] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:35.573318 9292 raft_consensus.cc:348] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.573459 9292 raft_consensus.cc:374] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:35.573515 9292 raft_consensus.cc:729] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21eb9cb925e5455fab6792b9008da016, State: Initialized, Role: FOLLOWER
I20260812 07:08:35.573671 9292 consensus_queue.cc:260] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.573990 9292 ts_tablet_manager.cc:1428] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 07:08:35.782862 9304 raft_consensus.cc:482] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:08:35.783046 9304 raft_consensus.cc:504] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.783679 9304 leader_election.cc:288] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745), 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283)
I20260812 07:08:35.784453 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a03ca6978a84476bced3ffdb665796d" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eb9cb925e5455fab6792b9008da016" is_pre_election: true
I20260812 07:08:35.784453 8869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a03ca6978a84476bced3ffdb665796d" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8333b15d0d914b158fe4df290765b3c2" is_pre_election: true
I20260812 07:08:35.784644 9036 raft_consensus.cc:2454] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 0.
I20260812 07:08:35.784644 8869 raft_consensus.cc:2454] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 0.
I20260812 07:08:35.785069 9120 leader_election.cc:302] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 21eb9cb925e5455fab6792b9008da016, 65cb8934dca145358c6f07283c2c91d8, 8333b15d0d914b158fe4df290765b3c2; no voters:
I20260812 07:08:35.785530 9304 raft_consensus.cc:2781] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:08:35.785702 9304 raft_consensus.cc:482] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:08:35.785802 9304 raft_consensus.cc:3037] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:35.787145 9304 raft_consensus.cc:504] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.787578 9304 leader_election.cc:288] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 election: Requested vote from peers 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745), 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283)
I20260812 07:08:35.788156 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a03ca6978a84476bced3ffdb665796d" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eb9cb925e5455fab6792b9008da016"
I20260812 07:08:35.788156 8869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a03ca6978a84476bced3ffdb665796d" candidate_uuid: "65cb8934dca145358c6f07283c2c91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8333b15d0d914b158fe4df290765b3c2"
I20260812 07:08:35.788342 9036 raft_consensus.cc:3037] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:35.788422 8869 raft_consensus.cc:3037] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:35.789584 8869 raft_consensus.cc:2454] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 1.
I20260812 07:08:35.789584 9036 raft_consensus.cc:2454] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65cb8934dca145358c6f07283c2c91d8 in term 1.
I20260812 07:08:35.789978 9120 leader_election.cc:302] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21eb9cb925e5455fab6792b9008da016, 65cb8934dca145358c6f07283c2c91d8; no voters:
I20260812 07:08:35.790200 9304 raft_consensus.cc:2781] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:35.790318 9304 raft_consensus.cc:686] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 1 LEADER]: Becoming Leader. State: Replica: 65cb8934dca145358c6f07283c2c91d8, State: Running, Role: LEADER
I20260812 07:08:35.790477 9304 consensus_queue.cc:237] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } }
I20260812 07:08:35.792816 8680 catalog_manager.cc:5495] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65cb8934dca145358c6f07283c2c91d8 (127.31.156.67). New cstate: current_term: 1 leader_uuid: "65cb8934dca145358c6f07283c2c91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65cb8934dca145358c6f07283c2c91d8" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35671 } health_report { overall_health: HEALTHY } } }
I20260812 07:08:35.830350 9379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:35.826612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:50401 (local address 127.31.156.67:35671)
0812 07:08:35.826822 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:35.826826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:35.827072 (+ 246us) server_negotiation.cc:407] Connection header received
0812 07:08:35.827278 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:35.827286 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:35.827330 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:35.827399 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:35.827784 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:35.828444 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:35.829334 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:35.829561 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:35.829647 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:35.829882 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:35.829953 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:35.830009 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:35.830126 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 07:08:35.830206 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":93}
I20260812 07:08:35.830363 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:35.826493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35671 (local address 127.31.156.67:50401)
0812 07:08:35.826953 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:35.826970 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:35.827085 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:35.827466 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:35.827469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:35.827477 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:08:35.827584 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:35.827590 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:35.828581 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:35.828584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:35.829200 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:35.829208 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:35.829672 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:35.829675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:35.829744 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:35.829775 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:08:35.830035 (+ 260us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:08:35.830041 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:08:35.830100 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 07:08:35.830192 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":2,"thread_start_us":237,"threads_started":1}
I20260812 07:08:35.837785 8869 raft_consensus.cc:1261] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Refusing update from remote peer 65cb8934dca145358c6f07283c2c91d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:35.837917 9036 raft_consensus.cc:1261] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Refusing update from remote peer 65cb8934dca145358c6f07283c2c91d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:35.838300 9304 consensus_queue.cc:1035] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8333b15d0d914b158fe4df290765b3c2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 39745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:35.838502 9398 consensus_queue.cc:1035] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21eb9cb925e5455fab6792b9008da016" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 45283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:35.871261 8680 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:35.871459 8680 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:08:35.873842 8680 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb677f7872c640c69cc53f045ffd73d2: Sending DeleteTablet for 3 replicas of tablet 5a03ca6978a84476bced3ffdb665796d
I20260812 07:08:35.874709 8841 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a03ca6978a84476bced3ffdb665796d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:35 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:36718
I20260812 07:08:35.874955 9392 tablet_replica.cc:330] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2: stopping tablet replica
I20260812 07:08:35.874928 9009 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a03ca6978a84476bced3ffdb665796d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:35 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:42366
I20260812 07:08:35.875104 9392 raft_consensus.cc:2229] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:35.875144 9391 tablet_replica.cc:330] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016: stopping tablet replica
I20260812 07:08:35.875247 9392 raft_consensus.cc:2258] T 5a03ca6978a84476bced3ffdb665796d P 8333b15d0d914b158fe4df290765b3c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:35.875264 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 8764
I20260812 07:08:35.875262 9391 raft_consensus.cc:2229] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:08:35.875428 9391 raft_consensus.cc:2258] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:35.876154 9391 ts_tablet_manager.cc:1905] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:35.878501 9391 ts_tablet_manager.cc:1918] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:08:35.878587 9391 log.cc:1196] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/5a03ca6978a84476bced3ffdb665796d
I20260812 07:08:35.878898 9391 ts_tablet_manager.cc:1939] T 5a03ca6978a84476bced3ffdb665796d P 21eb9cb925e5455fab6792b9008da016: Deleting consensus metadata
I20260812 07:08:35.879423 9180 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a03ca6978a84476bced3ffdb665796d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:08:35 UTC) from {username='oryx', principal='oryx/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:56344
I20260812 07:08:35.879660 9393 tablet_replica.cc:330] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8: stopping tablet replica
I20260812 07:08:35.879820 9393 raft_consensus.cc:2229] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:08:35.879854 8665 catalog_manager.cc:4841] TS 21eb9cb925e5455fab6792b9008da016 (127.31.156.66:45283): tablet 5a03ca6978a84476bced3ffdb665796d (table test-table [id=46ab38c360284d348baf4b7732038e49]) successfully deleted
I20260812 07:08:35.880069 9393 raft_consensus.cc:2258] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:08:35.881004 9393 ts_tablet_manager.cc:1905] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:08:35.884442 9393 ts_tablet_manager.cc:1918] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:08:35.884549 9393 log.cc:1196] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/5a03ca6978a84476bced3ffdb665796d
I20260812 07:08:35.884896 9393 ts_tablet_manager.cc:1939] T 5a03ca6978a84476bced3ffdb665796d P 65cb8934dca145358c6f07283c2c91d8: Deleting consensus metadata
I20260812 07:08:35.885771 8664 catalog_manager.cc:4841] TS 65cb8934dca145358c6f07283c2c91d8 (127.31.156.67:35671): tablet 5a03ca6978a84476bced3ffdb665796d (table test-table [id=46ab38c360284d348baf4b7732038e49]) successfully deleted
I20260812 07:08:35.893399 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 8933
W20260812 07:08:35.893388 8667 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:39745 (error 108)
I20260812 07:08:35.897403 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:35.895421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:39745 (local address 127.0.0.1:36742)
0812 07:08:35.897154 (+ 1733us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:35.897258 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:39745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1619,"thread_start_us":233,"threads_started":1}
W20260812 07:08:35.897610 8667 catalog_manager.cc:4584] TS 8333b15d0d914b158fe4df290765b3c2 (127.31.156.65:39745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a03ca6978a84476bced3ffdb665796d: Network error: Client connection negotiation failed: client connection to 127.31.156.65:39745: connect: Connection refused (error 111)
I20260812 07:08:35.905167 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 9101
I20260812 07:08:35.917025 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 8647
2026-08-12T07:08:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (4147 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:36 dist-test-slave-csg5 krb5kdc[9411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:36 dist-test-slave-csg5 krb5kdc[9411](info): set up 2 sockets
Aug 12 07:08:36 dist-test-slave-csg5 krb5kdc[9411](info): commencing operation
krb5kdc: starting...
W20260812 07:08:37.114830 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:08:37 dist-test-slave-csg5 krb5kdc[9411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:08:37.275786 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:39459
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:39459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:37.410512 9425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:37.411049 9425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:37.411149 9425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:37.416109 9425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:37.416208 9425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:37.416236 9425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:37.416254 9425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:37.416270 9425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:37.422523 9425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:39459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:39459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:37.423875 9425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:37.425055 9425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:37.435642 9434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:37.435667 9430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:37.435708 9425 server_base.cc:1034] running on GCE node
W20260812 07:08:37.436087 9431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:37.436982 9425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:37.438349 9425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:37.439626 9425 hybrid_clock.cc:648] HybridClock initialized: now 1786518517439600 us; error 41 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T07:08:37Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1521 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:37 dist-test-slave-csg5 krb5kdc[9440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:37 dist-test-slave-csg5 krb5kdc[9440](info): set up 2 sockets
Aug 12 07:08:37 dist-test-slave-csg5 krb5kdc[9440](info): commencing operation
krb5kdc: starting...
W20260812 07:08:38.651512 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:08:38 dist-test-slave-csg5 krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:08:38.807399 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:33891
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:33891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:38.930049 9454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:38.930392 9454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:38.930438 9454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:38.934753 9454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:38.934862 9454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:38.934882 9454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:38.934900 9454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:38.934916 9454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:38.940810 9454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:33891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:33891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:38.942138 9454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:38.943382 9454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:38.953419 9460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:38.953495 9461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:38.953513 9465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:38.953658 9454 server_base.cc:1034] running on GCE node
I20260812 07:08:38.954584 9454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:38.956017 9454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:38.957204 9454 hybrid_clock.cc:648] HybridClock initialized: now 1786518518957177 us; error 55 us; skew 500 ppm
Aug 12 07:08:38 dist-test-slave-csg5 krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518518, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:38.960783 9454 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:08:38.961905 9454 webserver.cc:460] Webserver started at http://127.31.156.126:33657/ using document root <none> and password file <none>
I20260812 07:08:38.962621 9454 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:38.962783 9454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:38.963057 9454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:38.965425 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "ec5b4a935b4a4651a00200ddece3414c"
format_stamp: "Formatted at 2026-08-12 07:08:38 on dist-test-slave-csg5"
server_key: "2d57054ced7bd2c16ca933aa3c0ac7e9"
server_key_iv: "79c915755c35ccfd59173a32b8d86984"
server_key_version: "encryptionkey@0"
I20260812 07:08:38.966084 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "ec5b4a935b4a4651a00200ddece3414c"
format_stamp: "Formatted at 2026-08-12 07:08:38 on dist-test-slave-csg5"
server_key: "2d57054ced7bd2c16ca933aa3c0ac7e9"
server_key_iv: "79c915755c35ccfd59173a32b8d86984"
server_key_version: "encryptionkey@0"
I20260812 07:08:38.970502 9454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260812 07:08:38.973521 9474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:38.975044 9454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:38.975176 9454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "ec5b4a935b4a4651a00200ddece3414c"
format_stamp: "Formatted at 2026-08-12 07:08:38 on dist-test-slave-csg5"
server_key: "2d57054ced7bd2c16ca933aa3c0ac7e9"
server_key_iv: "79c915755c35ccfd59173a32b8d86984"
server_key_version: "encryptionkey@0"
I20260812 07:08:38.975432 9454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:39.002741 9454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:39.006856 9454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:39.007110 9454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:39.016147 9454 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:33891
I20260812 07:08:39.016177 9538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:33891 every 8 connection(s)
I20260812 07:08:39.017234 9454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:39.020486 9539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:39.027060 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 9454
I20260812 07:08:39.027189 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:39.027273 9539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c: Bootstrap starting.
I20260812 07:08:39.027534 32369 external_mini_cluster.cc:1428] Setting key 077d2f66c751f8eb468319801620edc3
I20260812 07:08:39.030309 9539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:39.031234 9539 log.cc:824] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:39.033536 9539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c: No bootstrap required, opened a new log
Aug 12 07:08:39 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:39.038615 9539 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } }
I20260812 07:08:39.038977 9539 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:39.039062 9539 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5b4a935b4a4651a00200ddece3414c, State: Initialized, Role: FOLLOWER
I20260812 07:08:39.039667 9539 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } }
I20260812 07:08:39.039860 9539 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:39.039954 9539 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:39.040064 9539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:39.041630 9539 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } }
I20260812 07:08:39.042114 9539 leader_election.cc:302] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec5b4a935b4a4651a00200ddece3414c; no voters:
I20260812 07:08:39.042523 9539 leader_election.cc:288] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:39.042725 9546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:39.043067 9546 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 1 LEADER]: Becoming Leader. State: Replica: ec5b4a935b4a4651a00200ddece3414c, State: Running, Role: LEADER
I20260812 07:08:39.043479 9546 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } }
I20260812 07:08:39.044093 9539 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:39.045604 9550 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec5b4a935b4a4651a00200ddece3414c. Latest consensus state: current_term: 1 leader_uuid: "ec5b4a935b4a4651a00200ddece3414c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } } }
I20260812 07:08:39.045605 9547 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec5b4a935b4a4651a00200ddece3414c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } } }
I20260812 07:08:39.045799 9547 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:39.045799 9550 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:39.046484 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:39.029278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.31.156.126:33891)
0812 07:08:39.029858 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:39.029877 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:39.029918 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:08:39.030847 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:39.030870 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:39.031245 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:39.031653 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:39.032502 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.033731 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.034962 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.035521 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.035669 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:39.039361 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:39.039368 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:39.039388 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:39.039429 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:39.042059 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:39.043078 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:39.043089 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:39.043097 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:39.043271 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:39.043669 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:39.043672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:39.043674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:39.044278 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:39.044488 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:39.044851 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 07:08:39.045233 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":168,"threads_started":1}
I20260812 07:08:39.046536 9559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:39.050290 9559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:39.056751 9559 catalog_manager.cc:1347] Generated new cluster ID: 05847b8aeb1641e99128abf05a2e6ae9
I20260812 07:08:39.056854 9559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:39.098960 9559 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:39.100342 9559 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:39.106341 9559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c: Generated new TSK 0
I20260812 07:08:39.107280 9559 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:08:39.160341 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33891
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:39.289696 9574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:39.290058 9574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:39.290115 9574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:39.294914 9574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:39.295030 9574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:39.295097 9574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:39.301380 9574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:39.302731 9574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:39.303941 9574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:39.315825 9580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:39.315676 9582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:39.315907 9579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:39.315737 9574 server_base.cc:1034] running on GCE node
I20260812 07:08:39.316919 9574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:39.317713 9574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:39.319047 9574 hybrid_clock.cc:648] HybridClock initialized: now 1786518519319027 us; error 48 us; skew 500 ppm
Aug 12 07:08:39 dist-test-slave-csg5 krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518519, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:39.323750 9574 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:08:39.325033 9574 webserver.cc:460] Webserver started at http://127.31.156.65:45069/ using document root <none> and password file <none>
I20260812 07:08:39.325788 9574 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:39.325866 9574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:39.326068 9574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:39.328446 9574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "55ef106c61f8497397e5d5364a573241"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "2871f0683b38514df5dbf3f3575264ec"
server_key_iv: "f6ddc562057b9d102eda4aa182ce1d18"
server_key_version: "encryptionkey@0"
I20260812 07:08:39.329155 9574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "55ef106c61f8497397e5d5364a573241"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "2871f0683b38514df5dbf3f3575264ec"
server_key_iv: "f6ddc562057b9d102eda4aa182ce1d18"
server_key_version: "encryptionkey@0"
I20260812 07:08:39.333757 9574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:08:39.337177 9591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:39.339263 9574 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:08:39.339470 9574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "55ef106c61f8497397e5d5364a573241"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "2871f0683b38514df5dbf3f3575264ec"
server_key_iv: "f6ddc562057b9d102eda4aa182ce1d18"
server_key_version: "encryptionkey@0"
I20260812 07:08:39.339641 9574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:39.360776 9574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:39.365206 9574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:39.365480 9574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:39.366204 9574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:39.367368 9574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:39.367667 9574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:39.367863 9574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:39.367931 9574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:39.378966 9574 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:43149
I20260812 07:08:39.378989 9727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:43149 every 8 connection(s)
I20260812 07:08:39.380151 9574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:39.390128 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 9574
I20260812 07:08:39.390302 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:08:39.390750 32369 external_mini_cluster.cc:1428] Setting key 025bda4211127b67dff1d9d97d784ec6
Aug 12 07:08:39 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518519, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:39.397531 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:39.382440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:51607 (local address 127.31.156.126:33891)
0812 07:08:39.382619 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:39.382624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:39.383797 (+ 1173us) server_negotiation.cc:407] Connection header received
0812 07:08:39.385302 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:39.385305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:39.385365 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:39.385460 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:39.386933 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.387607 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.388952 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.389204 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.389312 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:39.392855 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:39.392858 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:39.392860 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:39.392889 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:39.394434 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:39.395292 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:39.395298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:39.395301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:39.395432 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:39.395966 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:39.395971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:39.395973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:39.396270 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:39.396390 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:39.397060 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 07:08:39.397244 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:08:39.398403 9730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:39.383040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:51607)
0812 07:08:39.383598 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:39.383649 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:39.384999 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:39.385672 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:39.385681 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:39.386142 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:39.386748 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:39.386764 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.387830 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.387833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:39.388791 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:39.388803 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.389391 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.389394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:39.389558 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:39.390365 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:39.390392 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:39.392615 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:39.394588 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:39.394599 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:39.394624 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:39.395122 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:39.395601 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:39.395605 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:39.395606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:39.395809 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:39.396408 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:39.396415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:39.396713 (+ 298us) client_negotiation.cc:769] Sending connection context
0812 07:08:39.396938 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 07:08:39.397308 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":154,"threads_started":1}
I20260812 07:08:39.400147 9728 heartbeater.cc:347] Connected to a master server at 127.31.156.126:33891
I20260812 07:08:39.400460 9728 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:39.401127 9728 heartbeater.cc:511] Master 127.31.156.126:33891 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:08:39.403110 9498 ts_manager.cc:194] Registered new tserver with Master: 55ef106c61f8497397e5d5364a573241 (127.31.156.65:43149)
I20260812 07:08:39.404564 9498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:51607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:08:39.432394 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33891
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:39.569000 9733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:39.569438 9733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:39.569533 9733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:39.574427 9733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:39.574579 9733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:39.574745 9733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:39.581099 9733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:39.582406 9733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:39.583657 9733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:39.596213 9739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:39.596232 9740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:39.596347 9733 server_base.cc:1034] running on GCE node
W20260812 07:08:39.596247 9742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:39.597025 9733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:39.597872 9733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:39.599051 9733 hybrid_clock.cc:648] HybridClock initialized: now 1786518519599040 us; error 40 us; skew 500 ppm
Aug 12 07:08:39 dist-test-slave-csg5 krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518519, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:39.602401 9733 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:08:39.603731 9733 webserver.cc:460] Webserver started at http://127.31.156.66:41601/ using document root <none> and password file <none>
I20260812 07:08:39.604453 9733 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:39.604566 9733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:39.604854 9733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:39.607906 9733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "4c2f772222e34693984736dabc561d45"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "36514a00b39b81a89e350a44bbb0f511"
server_key_iv: "9a2c7dbc59f003752a494fd657d5e217"
server_key_version: "encryptionkey@0"
I20260812 07:08:39.608609 9733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "4c2f772222e34693984736dabc561d45"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "36514a00b39b81a89e350a44bbb0f511"
server_key_iv: "9a2c7dbc59f003752a494fd657d5e217"
server_key_version: "encryptionkey@0"
I20260812 07:08:39.612972 9733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:39.616456 9753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:39.618026 9733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:39.618242 9733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "4c2f772222e34693984736dabc561d45"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "36514a00b39b81a89e350a44bbb0f511"
server_key_iv: "9a2c7dbc59f003752a494fd657d5e217"
server_key_version: "encryptionkey@0"
I20260812 07:08:39.618405 9733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:39.648916 9733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:39.652804 9733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:39.653059 9733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:39.653756 9733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:39.654727 9733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:39.654804 9733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:39.654902 9733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:39.654953 9733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:39.665966 9733 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:36161
I20260812 07:08:39.666018 9888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:36161 every 8 connection(s)
I20260812 07:08:39.667065 9733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:39.672489 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 9733
I20260812 07:08:39.672647 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:08:39.672926 32369 external_mini_cluster.cc:1428] Setting key 1c7b602a99b1ab82b41f206e919adf3b
Aug 12 07:08:39 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518519, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:39.685799 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:39.669360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:37927 (local address 127.31.156.126:33891)
0812 07:08:39.669566 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:39.669571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:39.670573 (+ 1002us) server_negotiation.cc:407] Connection header received
0812 07:08:39.672136 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:39.672140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:39.672218 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:39.672452 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:39.674139 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.674769 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.676655 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.676910 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.677008 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:39.680629 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:39.680633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:39.680635 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:39.680668 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:39.682409 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:39.683475 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:39.683479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:39.683480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:39.683582 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:39.684149 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:39.684152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:39.684153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:39.684399 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:39.684589 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:39.685494 (+ 905us) server_negotiation.cc:299] Negotiation successful
0812 07:08:39.685633 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:08:39.687009 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:39.669720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:37927)
0812 07:08:39.670379 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:39.670431 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:39.671879 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:39.672682 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:39.672692 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:39.673236 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:39.673964 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:39.673983 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.674938 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.674942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:39.676471 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:39.676489 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.677108 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.677115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:39.677406 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:39.678403 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:39.678424 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:39.680377 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:39.682594 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:39.682605 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:39.682632 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:39.683167 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:39.683724 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:39.683728 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:39.683730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:39.683986 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:39.684604 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:39.684612 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:39.685062 (+ 450us) client_negotiation.cc:769] Sending connection context
0812 07:08:39.685357 (+ 295us) client_negotiation.cc:240] Negotiation successful
0812 07:08:39.685815 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":137,"threads_started":1}
I20260812 07:08:39.689030 9889 heartbeater.cc:347] Connected to a master server at 127.31.156.126:33891
I20260812 07:08:39.689556 9889 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
I20260812 07:08:39.690251 9889 heartbeater.cc:511] Master 127.31.156.126:33891 requested a full tablet report, sending...
I20260812 07:08:39.691798 9498 ts_manager.cc:194] Registered new tserver with Master: 4c2f772222e34693984736dabc561d45 (127.31.156.66:36161)
I20260812 07:08:39.692765 9498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:37927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:08:39.716535 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33891
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:39.852552 9897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:39.852963 9897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:39.853067 9897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:39.858861 9897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:39.858986 9897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:39.859107 9897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:39.865723 9897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:39.867020 9897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:39.868175 9897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:08:39.879972 9897 server_base.cc:1034] running on GCE node
W20260812 07:08:39.879894 9902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:39.879890 9906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:39.879920 9903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:39.881012 9897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:39.881868 9897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:39.883080 9897 hybrid_clock.cc:648] HybridClock initialized: now 1786518519883049 us; error 43 us; skew 500 ppm
Aug 12 07:08:39 dist-test-slave-csg5 krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518519, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:39.886945 9897 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:08:39.888048 9897 webserver.cc:460] Webserver started at http://127.31.156.67:39123/ using document root <none> and password file <none>
I20260812 07:08:39.888710 9897 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:39.888821 9897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:39.889077 9897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:39.891495 9897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "883a282054614d1b836a5300d3161656"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "3860447bb047a1a7be83f241a152b1bf"
server_key_iv: "ccefd6965b8434464214b58b598833ef"
server_key_version: "encryptionkey@0"
I20260812 07:08:39.892172 9897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "883a282054614d1b836a5300d3161656"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "3860447bb047a1a7be83f241a152b1bf"
server_key_iv: "ccefd6965b8434464214b58b598833ef"
server_key_version: "encryptionkey@0"
I20260812 07:08:39.896986 9897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:08:39.900946 9913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:39.902915 9897 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:08:39.903055 9897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "883a282054614d1b836a5300d3161656"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "3860447bb047a1a7be83f241a152b1bf"
server_key_iv: "ccefd6965b8434464214b58b598833ef"
server_key_version: "encryptionkey@0"
I20260812 07:08:39.903193 9897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:39.915285 9897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:39.919180 9897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:39.919369 9897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:39.919976 9897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:39.920859 9897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:39.920910 9897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:39.920953 9897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:39.920969 9897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:39.931529 9897 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:40223
I20260812 07:08:39.931594 10047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:40223 every 8 connection(s)
I20260812 07:08:39.932670 9897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:39.936779 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 9897
I20260812 07:08:39.937043 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:08:39.937459 32369 external_mini_cluster.cc:1428] Setting key 124a6e519a6d8b8d94a9d86b8b789b95
Aug 12 07:08:39 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518519, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:39.950031 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:39.935383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:46379 (local address 127.31.156.126:33891)
0812 07:08:39.935552 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:39.935556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:39.936774 (+ 1218us) server_negotiation.cc:407] Connection header received
0812 07:08:39.938141 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:39.938145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:39.938211 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:39.938328 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:39.939742 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.940301 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.941590 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.941838 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.941958 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:39.945399 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:39.945409 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:39.945411 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:39.945440 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:39.947247 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:39.948087 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:39.948092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:39.948094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:39.948206 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:39.948682 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:39.948686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:39.948688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:39.948947 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:39.949073 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:39.949689 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 07:08:39.949840 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:08:39.951323 10051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:39.935622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:46379)
0812 07:08:39.936557 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:39.936606 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:39.937889 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:39.938536 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:39.938544 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:39.939032 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:39.939573 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:39.939587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.940474 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.940478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:39.941429 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:39.941440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:39.942027 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:39.942031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:39.942228 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:39.943064 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:39.943082 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:39.945022 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:39.947508 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:39.947516 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:39.947537 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:39.947949 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:39.948345 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:39.948349 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:39.948351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:39.948540 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:39.949073 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:39.949083 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:39.949399 (+ 316us) client_negotiation.cc:769] Sending connection context
0812 07:08:39.949637 (+ 238us) client_negotiation.cc:240] Negotiation successful
0812 07:08:39.950053 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":173,"threads_started":1}
I20260812 07:08:39.952996 10048 heartbeater.cc:347] Connected to a master server at 127.31.156.126:33891
I20260812 07:08:39.953392 10048 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:39.954021 10048 heartbeater.cc:511] Master 127.31.156.126:33891 requested a full tablet report, sending...
I20260812 07:08:39.955355 9498 ts_manager.cc:194] Registered new tserver with Master: 883a282054614d1b836a5300d3161656 (127.31.156.67:40223)
I20260812 07:08:39.955941 9498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:46379
I20260812 07:08:39.965318 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab.
Entry for principal oryx/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab.
WARNING: no policy specified for oryx/127.31.156.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab.
Entry for principal oryx/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab.
WARNING: no policy specified for oryx/127.31.156.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.156.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab.
Entry for principal oryx/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab.
I20260812 07:08:40.080019 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 9574
I20260812 07:08:40.091374 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 9733
I20260812 07:08:40.102109 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 9897
I20260812 07:08:40.111722 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 9454
I20260812 07:08:40.123383 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:33891
--webserver_interface=127.31.156.126
--webserver_port=33657
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:33891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:40.260834 10058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:40.261212 10058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:40.261260 10058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:40.266000 10058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:40.266111 10058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:40.266134 10058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:40.266150 10058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:40.266166 10058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:40.272451 10058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:33891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:33891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=33657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:40.273912 10058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:40.275245 10058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:40.286873 10065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:40.286890 10066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:40.287372 10058 server_base.cc:1034] running on GCE node
W20260812 07:08:40.286873 10068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:40.289263 10058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:40.291477 10058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:40.293027 10058 hybrid_clock.cc:648] HybridClock initialized: now 1786518520292988 us; error 46 us; skew 500 ppm
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518520, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:40.298241 10058 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:08:40.299710 10058 webserver.cc:460] Webserver started at http://127.31.156.126:33657/ using document root <none> and password file <none>
I20260812 07:08:40.300454 10058 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:40.300786 10058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:40.308034 10058 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:08:40.311185 10075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:40.313045 10058 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 07:08:40.313287 10058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "ec5b4a935b4a4651a00200ddece3414c"
format_stamp: "Formatted at 2026-08-12 07:08:38 on dist-test-slave-csg5"
server_key: "2d57054ced7bd2c16ca933aa3c0ac7e9"
server_key_iv: "79c915755c35ccfd59173a32b8d86984"
server_key_version: "encryptionkey@0"
I20260812 07:08:40.313984 10058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:40.339231 10058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:40.344277 10058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:40.344534 10058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:40.354900 10058 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:33891
I20260812 07:08:40.354924 10137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:33891 every 8 connection(s)
I20260812 07:08:40.356419 10058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:40.361438 10139 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:08:40.363516 10139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c: Bootstrap starting.
I20260812 07:08:40.364861 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 10058
I20260812 07:08:40.365458 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:43149
--local_ip_for_outbound_sockets=127.31.156.65
--tserver_master_addrs=127.31.156.126:33891
--webserver_port=45069
--webserver_interface=127.31.156.65
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.156.65
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:08:40.383872 10139 log.cc:824] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:40.391619 10139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:08:40.392112 10139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c: Bootstrap complete.
I20260812 07:08:40.396914 10139 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } }
I20260812 07:08:40.397416 10139 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5b4a935b4a4651a00200ddece3414c, State: Initialized, Role: FOLLOWER
I20260812 07:08:40.398029 10139 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } }
I20260812 07:08:40.398212 10139 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:40.398301 10139 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:40.398428 10139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:08:40.401235 10139 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } }
I20260812 07:08:40.401655 10139 leader_election.cc:302] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec5b4a935b4a4651a00200ddece3414c; no voters:
I20260812 07:08:40.402081 10139 leader_election.cc:288] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:08:40.402192 10143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:08:40.402441 10143 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [term 2 LEADER]: Becoming Leader. State: Replica: ec5b4a935b4a4651a00200ddece3414c, State: Running, Role: LEADER
I20260812 07:08:40.402832 10143 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } }
I20260812 07:08:40.403539 10139 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:40.404675 10145 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec5b4a935b4a4651a00200ddece3414c. Latest consensus state: current_term: 2 leader_uuid: "ec5b4a935b4a4651a00200ddece3414c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } } }
I20260812 07:08:40.404860 10145 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:40.405153 10144 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ec5b4a935b4a4651a00200ddece3414c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b4a935b4a4651a00200ddece3414c" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33891 } } }
I20260812 07:08:40.405251 10144 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:40.406613 10154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:40.409768 10154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:40.410445 10154 catalog_manager.cc:1259] Loaded cluster ID: 05847b8aeb1641e99128abf05a2e6ae9
I20260812 07:08:40.410526 10154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:40.412474 10154 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:40.413136 10154 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ec5b4a935b4a4651a00200ddece3414c: Loaded TSK: 0
I20260812 07:08:40.414193 10154 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:08:40.524374 10141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:40.524735 10141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:40.524822 10141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:40.529570 10141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:40.529719 10141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:40.529829 10141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:40.536581 10141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.65.keytab
--principal=oryx/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:43149
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=45069
--tserver_master_addrs=127.31.156.126:33891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:40.537878 10141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:40.539167 10141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:40.550444 10169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:40.550501 10168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:40.550706 10141 server_base.cc:1034] running on GCE node
W20260812 07:08:40.550845 10171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:40.551465 10141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:40.552440 10141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:40.553673 10141 hybrid_clock.cc:648] HybridClock initialized: now 1786518520553642 us; error 51 us; skew 500 ppm
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518520, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:40.557971 10141 init.cc:377] Logged in from keytab as oryx/127.31.156.65@KRBTEST.COM (short username oryx)
I20260812 07:08:40.559386 10141 webserver.cc:460] Webserver started at http://127.31.156.65:45069/ using document root <none> and password file <none>
I20260812 07:08:40.560097 10141 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:40.560249 10141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:40.566015 10141 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:08:40.569243 10178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:40.571038 10141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:40.571225 10141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "55ef106c61f8497397e5d5364a573241"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "2871f0683b38514df5dbf3f3575264ec"
server_key_iv: "f6ddc562057b9d102eda4aa182ce1d18"
server_key_version: "encryptionkey@0"
I20260812 07:08:40.571942 10141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:40.592001 10141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:40.597111 10141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:40.597328 10141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:40.598076 10141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:40.599143 10141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:40.599203 10141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:40.599267 10141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:40.599334 10141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:40.611685 10141 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:43149
I20260812 07:08:40.611845 10313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:43149 every 8 connection(s)
I20260812 07:08:40.613037 10141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:40.615954 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 10141
I20260812 07:08:40.616581 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:36161
--local_ip_for_outbound_sockets=127.31.156.66
--tserver_master_addrs=127.31.156.126:33891
--webserver_port=41601
--webserver_interface=127.31.156.66
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.156.66
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:08:40.630600 10316 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:40.615663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:59801 (local address 127.31.156.126:33891)
0812 07:08:40.616370 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:40.616410 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:40.617192 (+ 782us) server_negotiation.cc:407] Connection header received
0812 07:08:40.618884 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:40.618896 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:40.619239 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:40.619655 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:40.621039 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.622124 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.623560 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.624093 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.624297 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.629705 (+ 5408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:59801: BlockingRecv error: recv got EOF from 127.31.156.65:59801 (error 108)
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":252,"threads_started":1}
I20260812 07:08:40.630789 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:40.615914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:59801)
0812 07:08:40.616952 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:40.617043 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:40.618037 (+ 994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:40.619740 (+ 1703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:40.619749 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:40.620283 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:40.620808 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.620826 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.622307 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.622311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.623401 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.623415 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.624301 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.624305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.624506 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.625268 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:40.625293 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:40.629375 (+ 4082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":175,"threads_started":1}
W20260812 07:08:40.631541 10314 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:33891 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.156.126:33891: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:40.639477 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:40.632260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:58861)
0812 07:08:40.632509 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:40.632529 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:40.632646 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:40.633014 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:40.633017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:40.633202 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:40.633333 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.633342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.634583 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.634586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.635318 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.635330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.636041 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.636044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.636136 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.636817 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:40.636832 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:40.639262 (+ 2430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148}
W20260812 07:08:40.639855 10316 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:40.632484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:58861 (local address 127.31.156.126:33891)
0812 07:08:40.632643 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:40.632648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:40.632668 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:08:40.632802 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:40.632805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:40.632862 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:40.632963 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:40.633472 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.634441 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.635449 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.635881 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.636003 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.639544 (+ 3541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:58861: BlockingRecv error: recv got EOF from 127.31.156.65:58861 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:40.647379 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:40.640172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:34401)
0812 07:08:40.640444 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:40.640457 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:40.640556 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:40.640830 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:40.640833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:40.640982 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:40.641077 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.641081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.642308 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.642311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.643049 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.643061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.643736 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.643740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.643820 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.644434 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:40.644454 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:40.647224 (+ 2770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199}
W20260812 07:08:40.647706 10314 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:08:40.647756 10316 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:40.640259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:34401 (local address 127.31.156.126:33891)
0812 07:08:40.640397 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:40.640401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:40.640533 (+ 132us) server_negotiation.cc:407] Connection header received
0812 07:08:40.640651 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:40.640655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:40.640709 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:40.640803 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:40.641209 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.642161 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.643176 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.643577 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.643702 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.647615 (+ 3913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:34401: BlockingRecv error: recv got EOF from 127.31.156.65:34401 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 07:08:40.764389 10318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:40.764726 10318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:40.764775 10318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:40.769205 10318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:40.769304 10318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:40.769368 10318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:40.775322 10318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.66.keytab
--principal=oryx/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:36161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=41601
--tserver_master_addrs=127.31.156.126:33891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:40.776654 10318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:40.777879 10318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:40.789451 10323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:40.789446 10326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:40.789446 10324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:40.789564 10318 server_base.cc:1034] running on GCE node
I20260812 07:08:40.790186 10318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:40.791139 10318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:40.792337 10318 hybrid_clock.cc:648] HybridClock initialized: now 1786518520792313 us; error 38 us; skew 500 ppm
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518520, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:40.795861 10318 init.cc:377] Logged in from keytab as oryx/127.31.156.66@KRBTEST.COM (short username oryx)
I20260812 07:08:40.796890 10318 webserver.cc:460] Webserver started at http://127.31.156.66:41601/ using document root <none> and password file <none>
I20260812 07:08:40.797492 10318 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:40.797575 10318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:40.802367 10318 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:40.805050 10334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:40.806716 10318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:08:40.806859 10318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "4c2f772222e34693984736dabc561d45"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "36514a00b39b81a89e350a44bbb0f511"
server_key_iv: "9a2c7dbc59f003752a494fd657d5e217"
server_key_version: "encryptionkey@0"
I20260812 07:08:40.807451 10318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:40.821182 10318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:40.825618 10318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:40.825824 10318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:40.826496 10318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:40.827646 10318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:40.827728 10318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:40.827847 10318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:40.827875 10318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:40.839033 10318 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:36161
I20260812 07:08:40.839160 10466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:36161 every 8 connection(s)
I20260812 07:08:40.840312 10318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:40.846879 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 10318
I20260812 07:08:40.847420 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:40223
--local_ip_for_outbound_sockets=127.31.156.67
--tserver_master_addrs=127.31.156.126:33891
--webserver_port=39123
--webserver_interface=127.31.156.67
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.156.67
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:08:40.854254 10316 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:40.842573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:49787 (local address 127.31.156.126:33891)
0812 07:08:40.842765 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:40.842772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:40.844041 (+ 1269us) server_negotiation.cc:407] Connection header received
0812 07:08:40.845007 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:40.845012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:40.845081 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:40.845209 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:40.846456 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.847327 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.848966 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.849365 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.849495 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.854093 (+ 4598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:49787: BlockingRecv error: recv got EOF from 127.31.156.66:49787 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:08:40.855064 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:40.843046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:49787)
0812 07:08:40.843866 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:40.843912 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:40.844785 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:40.845392 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:40.845399 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:40.845801 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:40.846304 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.846317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.847493 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.847522 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.848799 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.848812 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.849569 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.849574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.849773 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.850691 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:40.850716 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:40.853935 (+ 3219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":140,"threads_started":1}
W20260812 07:08:40.855783 10468 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:33891 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.156.126:33891: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:40.862958 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:40.856408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:35475)
0812 07:08:40.856693 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:40.856707 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:40.856853 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:40.857177 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:40.857181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:40.857419 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:40.857543 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.857549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.858601 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.858606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.859304 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.859314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.859826 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.859830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.859934 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.860567 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:40.860579 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:40.862790 (+ 2211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":74}
W20260812 07:08:40.863106 10316 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:40.856531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:35475 (local address 127.31.156.126:33891)
0812 07:08:40.856648 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:40.856652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:40.856785 (+ 133us) server_negotiation.cc:407] Connection header received
0812 07:08:40.856998 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:40.857002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:40.857052 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:40.857157 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:40.857856 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.858466 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.859451 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.859689 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.859821 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.862955 (+ 3134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:35475: BlockingRecv error: recv got EOF from 127.31.156.66:35475 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:40 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:40.869505 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:40.863737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:50243)
0812 07:08:40.863864 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:40.863881 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:40.863967 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:40.864220 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:40.864223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:40.864396 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:40.864477 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.864482 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.865306 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.865309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.865859 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:40.865868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.866351 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.866355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:40.866437 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.867076 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:40.867100 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:40.869322 (+ 2222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:08:40.869664 10316 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:40.863842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:50243 (local address 127.31.156.126:33891)
0812 07:08:40.863958 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:40.863962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:40.863977 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:08:40.864055 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:40.864057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:40.864103 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:40.864202 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:40.864599 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.865170 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.865983 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:40.866228 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:40.866343 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:40.869499 (+ 3156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:50243: BlockingRecv error: recv got EOF from 127.31.156.66:50243 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 07:08:40.870016 10468 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:08:40.989339 10472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:40.989687 10472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:40.989739 10472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:40.994592 10472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:40.994709 10472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:40.994773 10472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:41.000620 10472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/krb5kdc/oryx_127.31.156.67.keytab
--principal=oryx/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:40223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=39123
--tserver_master_addrs=127.31.156.126:33891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:41.001833 10472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:41.003335 10472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:41.015630 10483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:41.016026 10472 server_base.cc:1034] running on GCE node
W20260812 07:08:41.015617 10480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:41.015617 10478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:41.016702 10472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:41.017544 10472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:41.018755 10472 hybrid_clock.cc:648] HybridClock initialized: now 1786518521018712 us; error 47 us; skew 500 ppm
Aug 12 07:08:41 dist-test-slave-csg5 krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518521, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:41.022143 10472 init.cc:377] Logged in from keytab as oryx/127.31.156.67@KRBTEST.COM (short username oryx)
I20260812 07:08:41.023447 10472 webserver.cc:460] Webserver started at http://127.31.156.67:39123/ using document root <none> and password file <none>
I20260812 07:08:41.024119 10472 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:41.024240 10472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:41.029460 10472 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:41.032450 10490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:41.034396 10472 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 07:08:41.034557 10472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "883a282054614d1b836a5300d3161656"
format_stamp: "Formatted at 2026-08-12 07:08:39 on dist-test-slave-csg5"
server_key: "3860447bb047a1a7be83f241a152b1bf"
server_key_iv: "ccefd6965b8434464214b58b598833ef"
server_key_version: "encryptionkey@0"
I20260812 07:08:41.035360 10472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:41.061821 10472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:41.065963 10472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:41.066162 10472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:41.066861 10472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:41.067778 10472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:41.067830 10472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:41.067873 10472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:41.067888 10472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:41.078434 10472 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:40223
I20260812 07:08:41.078459 10628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:40223 every 8 connection(s)
I20260812 07:08:41.079666 10472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:41.088440 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 10472
Aug 12 07:08:41 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:41 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:08:41.093731 10316 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:41.081836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:40477 (local address 127.31.156.126:33891)
0812 07:08:41.082026 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:41.082031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:41.083401 (+ 1370us) server_negotiation.cc:407] Connection header received
0812 07:08:41.084326 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:41.084330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:41.084385 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:41.084484 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:41.086012 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.086603 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.088553 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.088957 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.089072 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:41.093585 (+ 4513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:40477: BlockingRecv error: recv got EOF from 127.31.156.67:40477 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 07:08:41.094501 10633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:41.082333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:40477)
0812 07:08:41.083225 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:41.083274 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:41.084086 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:41.084767 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:41.084776 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:41.085232 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:41.085813 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:41.085830 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.086949 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.086952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:41.088380 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:41.088395 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.089194 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.089201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:41.089463 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:41.090352 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:41.090376 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:41.093464 (+ 3088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":139,"threads_started":1}
W20260812 07:08:41.095147 10630 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:33891 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.156.126:33891: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
I20260812 07:08:41.101339 10635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:41.089702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.31.156.126:33891)
0812 07:08:41.090057 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:41.090064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:41.090081 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:08:41.090130 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:41.090134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:41.090212 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:41.090310 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:41.091438 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.092067 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.093156 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.093402 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.093520 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:41.095459 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:41.095469 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:41.095491 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:41.095535 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:41.097951 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:41.098584 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:41.098590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:41.098596 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:41.098906 (+ 310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:41.099309 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:41.099312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:41.099314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:41.099910 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:41.100184 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:41.100717 (+ 533us) server_negotiation.cc:299] Negotiation successful
0812 07:08:41.101073 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":121,"threads_started":1}
Aug 12 07:08:41 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:41 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:41.102984 10633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:41.095709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:43301)
0812 07:08:41.096106 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:41.096129 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:41.096353 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:41.096655 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:41.096658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:41.096857 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:41.096966 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:41.096972 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.097979 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.097982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:41.098942 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:41.098953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.099668 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.099671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:41.099769 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:41.100365 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:41.100383 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:41.102867 (+ 2484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306}
W20260812 07:08:41.103156 10316 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:41.095913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:43301 (local address 127.31.156.126:33891)
0812 07:08:41.096070 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:41.096075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:41.096205 (+ 130us) server_negotiation.cc:407] Connection header received
0812 07:08:41.096485 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:41.096488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:41.096542 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:41.096636 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:41.097094 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.097791 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.099078 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.099516 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.099649 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:41.103038 (+ 3389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:43301: BlockingRecv error: recv got EOF from 127.31.156.67:43301 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 07:08:41 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:41 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:41.110618 10633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:41.103885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:43073)
0812 07:08:41.104049 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:41.104068 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:41.104152 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:41.104506 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:41.104512 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:41.104753 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:41.104864 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:41.104875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.105862 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.105865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:41.106431 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:41.106440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.107198 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.107201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:41.107270 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:41.107884 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:41.107898 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:41.110402 (+ 2504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 07:08:41.110782 10316 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:41.104009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:43073 (local address 127.31.156.126:33891)
0812 07:08:41.104160 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:41.104164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:41.104178 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:08:41.104248 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:41.104251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:41.104299 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:41.104391 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:41.104979 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.105684 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.106589 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.107036 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.107149 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:41.110595 (+ 3446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:43073: BlockingRecv error: recv got EOF from 127.31.156.67:43073 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 07:08:41.111023 10630 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 07:08:41 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:41 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:41.655380 10636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:41.648435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:32775)
0812 07:08:41.648745 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:41.648763 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:41.648882 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:41.649266 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:41.649269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:41.649493 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:41.649630 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:41.649636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.650554 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.650558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:41.651359 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:41.651372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.651903 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.651907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:41.651996 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:41.652683 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:41.652697 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:41.655143 (+ 2446us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
W20260812 07:08:41.655642 10637 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:41.648577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:32775 (local address 127.31.156.126:33891)
0812 07:08:41.648889 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:41.648893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:41.648907 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:08:41.648962 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:41.648964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:41.649009 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:41.649081 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:41.649812 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.650413 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.651508 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.651761 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.651888 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:41.655467 (+ 3579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:32775: BlockingRecv error: recv got EOF from 127.31.156.65:32775 (error 108)
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":138,"threads_started":1}
Aug 12 07:08:41 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:41 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:41.877835 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:41.870875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:56711)
0812 07:08:41.871307 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:41.871320 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:41.871478 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:41.871830 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:41.871833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:41.872063 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:41.872194 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:41.872200 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.872981 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.872985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:41.873721 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:41.873734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.874436 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.874440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:41.874535 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:41.875298 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:41.875320 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:41.877707 (+ 2387us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":78,"threads_started":1}
W20260812 07:08:41.878050 10637 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:41.870959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:56711 (local address 127.31.156.126:33891)
0812 07:08:41.871136 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:41.871141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:41.871413 (+ 272us) server_negotiation.cc:407] Connection header received
0812 07:08:41.871604 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:41.871607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:41.871656 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:41.871802 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:41.872323 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.872861 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.873876 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:41.874245 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:41.874399 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:41.877869 (+ 3470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:56711: BlockingRecv error: recv got EOF from 127.31.156.66:56711 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 12 07:08:42 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:42 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:42.118894 10640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:42.111943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:56131)
0812 07:08:42.112287 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:42.112306 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:42.112477 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:42.112867 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:42.112871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:42.113117 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:42.113225 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:42.113232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.114222 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.114228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:42.114945 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:42.114957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.115494 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.115498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:42.115596 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:42.116256 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:42.116277 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:42.118691 (+ 2414us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":93,"threads_started":1}
W20260812 07:08:42.119024 10637 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:42.112124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:56131 (local address 127.31.156.126:33891)
0812 07:08:42.112279 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:42.112284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:42.112403 (+ 119us) server_negotiation.cc:407] Connection header received
0812 07:08:42.112602 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:42.112606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:42.112707 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:42.112835 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:42.113441 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.114084 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.115087 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.115355 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.115483 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:42.118887 (+ 3404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:56131: BlockingRecv error: recv got EOF from 127.31.156.67:56131 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 07:08:42 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:42 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:42.663352 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:42.656353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:34597)
0812 07:08:42.656732 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:42.656752 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:42.656856 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:42.657370 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:42.657373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:42.657574 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:42.657680 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:42.657685 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.658547 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.658550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:42.659184 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:42.659195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.659822 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.659825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:42.659894 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:42.660575 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:42.660588 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:42.663196 (+ 2608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":104,"threads_started":1}
W20260812 07:08:42.663684 10642 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:42.656513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:34597 (local address 127.31.156.126:33891)
0812 07:08:42.656790 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:42.656794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:42.656809 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:08:42.657003 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:42.657005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:42.657056 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:42.657167 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:42.657809 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.658423 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.659396 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.659683 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.659792 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:42.663428 (+ 3636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:34597: BlockingRecv error: recv got EOF from 127.31.156.65:34597 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":96,"threads_started":1}
Aug 12 07:08:42 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:42 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:42.885726 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:42.878765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:40937)
0812 07:08:42.879089 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:42.879107 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:42.879236 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:42.879703 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:42.879707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:42.880136 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:42.880260 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:42.880267 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.881255 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.881259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:42.881870 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:42.881881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.882554 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.882558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:42.882632 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:42.883458 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:42.883473 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:42.885571 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":113,"threads_started":1}
W20260812 07:08:42.886027 10642 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:42.878833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:40937 (local address 127.31.156.126:33891)
0812 07:08:42.879003 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:42.879008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:42.879419 (+ 411us) server_negotiation.cc:407] Connection header received
0812 07:08:42.879468 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:42.879472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:42.879542 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:42.879617 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:42.880449 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.881012 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.882031 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:42.882404 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:42.882520 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:42.885894 (+ 3374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:40937: BlockingRecv error: recv got EOF from 127.31.156.66:40937 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 07:08:43 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:43 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:43.126940 10645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:43.120004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:57515)
0812 07:08:43.120439 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:43.120453 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:43.120556 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:43.120848 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:43.120852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:43.121086 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:43.121290 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:43.121298 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.122248 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.122251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:43.123113 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:43.123126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.123593 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.123596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:43.123679 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:43.124227 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:43.124240 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:43.126779 (+ 2539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":135,"threads_started":1}
W20260812 07:08:43.127152 10642 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:43.120129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:57515 (local address 127.31.156.126:33891)
0812 07:08:43.120271 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:43.120278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:43.120527 (+ 249us) server_negotiation.cc:407] Connection header received
0812 07:08:43.120625 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:43.120627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:43.120674 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:43.120750 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:43.121474 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.122037 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.123241 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.123472 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.123556 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:43.126993 (+ 3437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:57515: BlockingRecv error: recv got EOF from 127.31.156.67:57515 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 07:08:43 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:43 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:43.676097 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:43.664394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:45617)
0812 07:08:43.664775 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:43.664791 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:43.664892 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:43.665199 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:43.665201 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:43.665493 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:43.665605 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:43.665612 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.666525 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.666528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:43.667231 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:43.667242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.667816 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.667819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:43.667909 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:43.673279 (+ 5370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:43.673299 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:43.675898 (+ 2599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":113,"threads_started":1}
W20260812 07:08:43.676230 10647 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:43.664546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:45617 (local address 127.31.156.126:33891)
0812 07:08:43.664863 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:43.664866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:43.664880 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:08:43.664965 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:43.664967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:43.665011 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:43.665088 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:43.665794 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.666391 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.667395 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.667660 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.667779 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:43.676088 (+ 8309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:45617: BlockingRecv error: recv got EOF from 127.31.156.65:45617 (error 108)
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":133,"threads_started":1}
Aug 12 07:08:43 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:43 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:43.893559 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:43.886964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:34303)
0812 07:08:43.887318 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:43.887334 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:43.887499 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:43.887798 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:43.887801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:43.888072 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:43.888208 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:43.888217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.889217 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.889220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:43.889863 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:43.889873 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.890364 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.890368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:43.890455 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:43.891086 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:43.891099 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:43.893412 (+ 2313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":101,"threads_started":1}
W20260812 07:08:43.893787 10647 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:43.887019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:34303 (local address 127.31.156.126:33891)
0812 07:08:43.887202 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:43.887207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:43.887428 (+ 221us) server_negotiation.cc:407] Connection header received
0812 07:08:43.887621 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:43.887624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:43.887683 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:43.887825 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:43.888345 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.889101 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.889992 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:43.890240 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:43.890357 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:43.893637 (+ 3280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:34303: BlockingRecv error: recv got EOF from 127.31.156.66:34303 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 07:08:44 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:44 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:44.134516 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:44.127958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:33727)
0812 07:08:44.128277 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:44.128293 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:44.128528 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:44.128832 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:44.128835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:44.129064 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:44.129165 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:44.129172 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.129961 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.129966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:44.130552 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:44.130561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.131079 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.131083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:44.131174 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:44.131886 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:44.131899 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:44.134298 (+ 2399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":92,"threads_started":1}
W20260812 07:08:44.134768 10647 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:44.128018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:33727 (local address 127.31.156.126:33891)
0812 07:08:44.128170 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:44.128175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:44.128461 (+ 286us) server_negotiation.cc:407] Connection header received
0812 07:08:44.128650 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:44.128654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:44.128713 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:44.128805 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:44.129292 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.129839 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.130705 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.130952 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.131071 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:44.134565 (+ 3494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:33727: BlockingRecv error: recv got EOF from 127.31.156.67:33727 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 07:08:44 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:44 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:44.684722 10651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:44.677538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:48389)
0812 07:08:44.677902 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:44.677929 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:44.678072 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:44.678467 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:44.678471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:44.678800 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:44.678951 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:44.678959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.679847 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.679852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:44.680516 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:44.680527 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.681081 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.681085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:44.681187 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:44.681824 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:44.681838 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:44.684508 (+ 2670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":151,"threads_started":1}
W20260812 07:08:44.684870 10652 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:44.677655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:48389 (local address 127.31.156.126:33891)
0812 07:08:44.677949 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:44.677956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:44.677984 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:08:44.678192 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:44.678195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:44.678249 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:44.678382 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:44.679103 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.679703 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.680661 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.680932 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.681072 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:44.684708 (+ 3636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:48389: BlockingRecv error: recv got EOF from 127.31.156.65:48389 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":90,"threads_started":1}
Aug 12 07:08:44 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:44 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:44.902133 10653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:44.894812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:45211)
0812 07:08:44.895409 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:44.895425 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:44.895615 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:44.895983 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:44.895986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:44.896274 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:44.896458 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:44.896468 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.897761 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.897764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:44.898367 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:44.898379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.898979 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.898982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:44.899071 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:44.899702 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:44.899715 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:44.901889 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":126,"threads_started":1}
W20260812 07:08:44.902174 10652 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:44.894826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:45211 (local address 127.31.156.126:33891)
0812 07:08:44.895123 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:44.895129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:44.895539 (+ 410us) server_negotiation.cc:407] Connection header received
0812 07:08:44.895751 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:44.895754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:44.895819 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:44.895906 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:44.896614 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.897603 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.898510 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:44.898851 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:44.898969 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:44.902067 (+ 3098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:45211: BlockingRecv error: recv got EOF from 127.31.156.66:45211 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 12 07:08:45 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:45 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:45.143622 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:45.135689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:42015)
0812 07:08:45.136183 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:45.136201 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:45.136342 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:45.136686 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:45.136690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:45.136939 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:45.137094 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:45.137104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.138293 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.138298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:45.139280 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:45.139296 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.139971 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.139976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:45.140112 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:45.140951 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:45.140965 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:45.143468 (+ 2503us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":99,"threads_started":1}
W20260812 07:08:45.143849 10652 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:45.135801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:42015 (local address 127.31.156.126:33891)
0812 07:08:45.135981 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:45.135987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:45.136321 (+ 334us) server_negotiation.cc:407] Connection header received
0812 07:08:45.136461 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:45.136465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:45.136530 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:45.136630 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:45.137251 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.138144 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.139427 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.139740 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.139830 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:45.143639 (+ 3809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:42015: BlockingRecv error: recv got EOF from 127.31.156.67:42015 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 07:08:45 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:45 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:45.693887 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:45.686001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:53143)
0812 07:08:45.686351 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:45.686401 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:45.686543 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:45.686938 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:45.686941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:45.687165 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:45.687294 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:45.687300 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.688502 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.688505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:45.689385 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:45.689400 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.690250 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.690261 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:45.690347 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:45.691105 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:45.691120 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:45.693741 (+ 2621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":96,"threads_started":1}
W20260812 07:08:45.694129 10656 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:45.686046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:53143 (local address 127.31.156.126:33891)
0812 07:08:45.686371 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:45.686377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:45.686493 (+ 116us) server_negotiation.cc:407] Connection header received
0812 07:08:45.686685 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:45.686689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:45.686758 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:45.686879 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:45.687548 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.688266 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.689742 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.690018 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.690148 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:45.693940 (+ 3792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:53143: BlockingRecv error: recv got EOF from 127.31.156.65:53143 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":70,"threads_started":1}
Aug 12 07:08:45 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:45 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:45.910835 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:45.903527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:39929)
0812 07:08:45.903914 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:45.903928 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:45.904036 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:45.904560 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:45.904563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:45.904831 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:45.904946 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:45.904952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.905927 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.905930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:45.906606 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:45.906618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.907483 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.907486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:45.907566 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:45.908325 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:45.908340 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:45.910612 (+ 2272us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":158,"threads_started":1}
W20260812 07:08:45.911150 10656 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:45.903644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:39929 (local address 127.31.156.126:33891)
0812 07:08:45.903812 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:45.903817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:45.904034 (+ 217us) server_negotiation.cc:407] Connection header received
0812 07:08:45.904110 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:45.904112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:45.904231 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:45.904488 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:45.905101 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.905787 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.906892 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:45.907168 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:45.907295 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:45.910961 (+ 3666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:39929: BlockingRecv error: recv got EOF from 127.31.156.66:39929 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 07:08:46 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:46 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:46.152043 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:46.145081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:48105)
0812 07:08:46.145611 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:46.145633 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:46.145850 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:46.146164 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:46.146168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:46.146407 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:46.146518 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:46.146524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.147488 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.147491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:46.148142 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:46.148153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.148757 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.148760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:46.148846 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:46.149451 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:46.149464 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:46.151893 (+ 2429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":111,"threads_started":1}
W20260812 07:08:46.152379 10656 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:46.145047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:48105 (local address 127.31.156.126:33891)
0812 07:08:46.145346 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:46.145376 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:46.145750 (+ 374us) server_negotiation.cc:407] Connection header received
0812 07:08:46.145974 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:46.145977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:46.146033 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:46.146109 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:46.146636 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.147339 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.148296 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.148547 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.148712 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:46.152062 (+ 3350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:48105: BlockingRecv error: recv got EOF from 127.31.156.67:48105 (error 108)
Metrics: {"server-negotiator.queue_time_us":197}
Aug 12 07:08:46 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:46 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:46.702405 10660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:46.695014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:38643)
0812 07:08:46.695445 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:46.695461 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:46.695588 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:46.696063 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:46.696067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:46.696493 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:46.696641 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:46.696651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.697772 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.697776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:46.698581 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:46.698595 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.699283 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.699286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:46.699366 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:46.699999 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:46.700012 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:46.702250 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":129,"threads_started":1}
W20260812 07:08:46.702715 10661 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:46.695142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:38643 (local address 127.31.156.126:33891)
0812 07:08:46.695473 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:46.695477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:46.695595 (+ 118us) server_negotiation.cc:407] Connection header received
0812 07:08:46.695726 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:46.695729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:46.695786 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:46.695877 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:46.696856 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.697606 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.698779 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.699086 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.699209 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:46.702532 (+ 3323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:38643: BlockingRecv error: recv got EOF from 127.31.156.65:38643 (error 108)
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":83,"threads_started":1}
Aug 12 07:08:46 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:46 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:46.920027 10662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:46.912311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:53517)
0812 07:08:46.912650 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:46.912670 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:46.912814 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:46.913169 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:46.913172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:46.913819 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:46.913945 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:46.913953 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.915126 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.915131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:46.915801 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:46.915814 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.916368 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.916371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:46.916440 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:46.917107 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:46.917122 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:46.919888 (+ 2766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":96,"threads_started":1}
W20260812 07:08:46.920289 10661 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:46.912338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:53517 (local address 127.31.156.126:33891)
0812 07:08:46.912514 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:46.912518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:46.912841 (+ 323us) server_negotiation.cc:407] Connection header received
0812 07:08:46.912908 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:46.912914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:46.912983 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:46.913065 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:46.914108 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.914967 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.915952 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:46.916185 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:46.916343 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:46.920103 (+ 3760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:53517: BlockingRecv error: recv got EOF from 127.31.156.66:53517 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 07:08:47 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:47 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:47.159950 10663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:47.153119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:57001)
0812 07:08:47.153721 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:47.153737 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:47.153882 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:47.154182 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:47.154185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:47.154434 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:47.154536 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:47.154542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.155363 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.155369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:47.155971 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:47.155980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.156477 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.156480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:47.156568 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:47.157172 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:47.157185 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:47.159698 (+ 2513us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":298,"threads_started":1}
W20260812 07:08:47.160017 10661 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:47.153375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:57001 (local address 127.31.156.126:33891)
0812 07:08:47.153595 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:47.153600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:47.153823 (+ 223us) server_negotiation.cc:407] Connection header received
0812 07:08:47.153993 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:47.153997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:47.154053 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:47.154158 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:47.154694 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.155242 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.156108 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.156347 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.156465 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:47.159889 (+ 3424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:57001: BlockingRecv error: recv got EOF from 127.31.156.67:57001 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 12 07:08:47 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:47 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:47.711251 10666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:47.703607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:36533)
0812 07:08:47.704050 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:47.704069 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:47.704201 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:47.704708 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:47.704712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:47.704924 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:47.705034 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:47.705062 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.706084 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.706088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:47.707038 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:47.707054 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.707717 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.707720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:47.707802 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:47.708429 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:47.708444 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:47.711090 (+ 2646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":97,"threads_started":1}
W20260812 07:08:47.711723 10665 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:47.703691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:36533 (local address 127.31.156.126:33891)
0812 07:08:47.703983 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:47.703988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:47.704163 (+ 175us) server_negotiation.cc:407] Connection header received
0812 07:08:47.704415 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:47.704431 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:47.704512 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:47.704608 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:47.705251 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.705936 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.707201 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.707500 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.707622 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:47.711507 (+ 3885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:36533: BlockingRecv error: recv got EOF from 127.31.156.65:36533 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":94,"threads_started":1}
Aug 12 07:08:47 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:47 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:47.927965 10667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:47.921153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:57959)
0812 07:08:47.921598 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:47.921618 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:47.921740 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:47.922013 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:47.922015 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:47.922209 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:47.922318 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:47.922323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.923224 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.923227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:47.923814 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:47.923823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.924339 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.924342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:47.924445 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:47.925185 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:47.925199 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:47.927785 (+ 2586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":118,"threads_started":1}
W20260812 07:08:47.928130 10665 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:47.921317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:57959 (local address 127.31.156.126:33891)
0812 07:08:47.921534 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:47.921538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:47.921730 (+ 192us) server_negotiation.cc:407] Connection header received
0812 07:08:47.921846 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:47.921849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:47.921903 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:47.921975 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:47.922442 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.923087 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.923948 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:47.924201 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:47.924293 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:47.927991 (+ 3698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:57959: BlockingRecv error: recv got EOF from 127.31.156.66:57959 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 12 07:08:48 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:48 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:48.168300 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:48.161155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:33611)
0812 07:08:48.161653 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:48.161667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:48.161871 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:48.162178 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:48.162182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:48.162409 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:48.162513 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:48.162520 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.163379 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.163383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:48.164058 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:48.164070 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.164567 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.164571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:48.164663 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:48.165345 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:48.165366 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:48.168159 (+ 2793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":224,"threads_started":1}
W20260812 07:08:48.168551 10665 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:48.161260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:33611 (local address 127.31.156.126:33891)
0812 07:08:48.161518 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:48.161525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:48.161757 (+ 232us) server_negotiation.cc:407] Connection header received
0812 07:08:48.161990 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:48.161998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:48.162050 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:48.162140 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:48.162691 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.163240 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.164181 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.164432 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.164550 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:48.168386 (+ 3836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:33611: BlockingRecv error: recv got EOF from 127.31.156.67:33611 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 12 07:08:48 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:48 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:48.730811 10671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:48.712491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:56295)
0812 07:08:48.712802 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:48.712817 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:48.712923 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:48.713382 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:48.713385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:48.713595 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:48.713783 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:48.713794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.715122 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.715126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:48.715825 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:48.715840 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.716447 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.716450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:48.716532 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:48.727255 (+ 10723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:48.727274 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:48.730541 (+ 3267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":106,"threads_started":1}
W20260812 07:08:48.730923 10672 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:48.712709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:56295 (local address 127.31.156.126:33891)
0812 07:08:48.712957 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:48.712961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:48.712976 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:08:48.713022 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:48.713024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:48.713066 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:48.713266 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:48.713990 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.714949 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.715996 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.716284 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.716393 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:48.730759 (+ 14366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:56295: BlockingRecv error: recv got EOF from 127.31.156.65:56295 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":89,"threads_started":1}
Aug 12 07:08:48 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:48 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:48.936080 10673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:48.929108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:44867)
0812 07:08:48.929504 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:48.929523 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:48.929680 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:48.929960 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:48.929963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:48.930220 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:48.930332 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:48.930339 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.931313 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.931320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:48.931943 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:48.931952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.932443 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.932447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:48.932548 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:48.933170 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:48.933184 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:48.935936 (+ 2752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":105,"threads_started":1}
W20260812 07:08:48.936446 10672 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:48.929137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:44867 (local address 127.31.156.126:33891)
0812 07:08:48.929296 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:48.929301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:48.929603 (+ 302us) server_negotiation.cc:407] Connection header received
0812 07:08:48.929786 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:48.929789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:48.929841 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:48.929934 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:48.930451 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.931197 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.932073 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:48.932316 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:48.932432 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:48.936307 (+ 3875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:44867: BlockingRecv error: recv got EOF from 127.31.156.66:44867 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 07:08:49 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:49 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:49.175799 10674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:49.169279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:53399)
0812 07:08:49.169610 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:49.169631 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:49.169761 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:49.170135 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:49.170138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:49.170339 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:49.170445 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:49.170451 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.171399 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.171402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:49.172065 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:49.172076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.172583 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.172587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:49.172660 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:49.173407 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:49.173421 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:49.175665 (+ 2244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":106,"threads_started":1}
W20260812 07:08:49.176016 10672 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:49.169263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:53399 (local address 127.31.156.126:33891)
0812 07:08:49.169486 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:49.169491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:49.169799 (+ 308us) server_negotiation.cc:407] Connection header received
0812 07:08:49.169897 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:49.169900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:49.169991 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:49.170070 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:49.170576 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.171266 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.172189 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.172435 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.172531 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:49.175863 (+ 3332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:53399: BlockingRecv error: recv got EOF from 127.31.156.67:53399 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Aug 12 07:08:49 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:49 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:49.738704 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:49.731935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:49221)
0812 07:08:49.732313 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:49.732328 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:49.732470 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:49.732748 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:49.732751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:49.733003 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:49.733111 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:49.733117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.733950 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.733953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:49.734514 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:49.734522 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.735120 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.735125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:49.735223 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:49.735920 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:49.735941 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:49.738474 (+ 2533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":104,"threads_started":1}
W20260812 07:08:49.738842 10677 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:49.732087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:49221 (local address 127.31.156.126:33891)
0812 07:08:49.732357 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:49.732361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:49.732416 (+ 55us) server_negotiation.cc:407] Connection header received
0812 07:08:49.732566 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:49.732569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:49.732616 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:49.732702 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:49.733227 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.733832 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.734683 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.734979 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.735073 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:49.738626 (+ 3553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:49221: BlockingRecv error: recv got EOF from 127.31.156.65:49221 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":67,"threads_started":1}
Aug 12 07:08:49 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:49 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:49.944484 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:49.937188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:54487)
0812 07:08:49.937532 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:49.937557 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:49.937859 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:49.938215 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:49.938218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:49.938477 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:49.938583 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:49.938590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.939522 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.939527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:49.940202 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:49.940213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.940709 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.940713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:49.940820 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:49.941482 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:49.941495 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:49.944182 (+ 2687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":97,"threads_started":1}
W20260812 07:08:49.944489 10677 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:49.937429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:54487 (local address 127.31.156.126:33891)
0812 07:08:49.937597 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:49.937604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:49.937642 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:08:49.938030 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:49.938034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:49.938092 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:49.938183 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:49.938770 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.939397 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.940336 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:49.940577 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:49.940686 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:49.944382 (+ 3696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:54487: BlockingRecv error: recv got EOF from 127.31.156.66:54487 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 07:08:50 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:50 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:50.184832 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:50.177167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:40695)
0812 07:08:50.177533 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:50.177550 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:50.177718 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:50.178064 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:50.178067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:50.178355 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:50.178495 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:50.178504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.179582 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.179585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:50.180191 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:50.180203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.180866 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.180869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:50.180946 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:50.181635 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:50.181649 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:50.184561 (+ 2912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":126,"threads_started":1}
W20260812 07:08:50.184881 10677 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:50.177242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:40695 (local address 127.31.156.126:33891)
0812 07:08:50.177424 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:50.177430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:50.177645 (+ 215us) server_negotiation.cc:407] Connection header received
0812 07:08:50.177838 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:50.177842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:50.177899 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:50.177995 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:50.178768 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.179448 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.180406 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.180696 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.180806 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:50.184731 (+ 3925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:40695: BlockingRecv error: recv got EOF from 127.31.156.67:40695 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 07:08:50 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:50 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:50.747300 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:50.739996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:34791)
0812 07:08:50.740420 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:50.740439 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:50.740662 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:50.741038 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:50.741045 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:50.741351 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:50.741497 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:50.741507 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.742534 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.742537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:50.743167 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:50.743178 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.743764 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.743767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:50.743845 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:50.744713 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:50.744727 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:50.747085 (+ 2358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":96,"threads_started":1}
W20260812 07:08:50.747529 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:50.740066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:34791 (local address 127.31.156.126:33891)
0812 07:08:50.740386 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:50.740392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:50.740556 (+ 164us) server_negotiation.cc:407] Connection header received
0812 07:08:50.740776 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:50.740780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:50.740850 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:50.740949 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:50.741728 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.742383 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.743330 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.743586 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.743691 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:50.747328 (+ 3637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:34791: BlockingRecv error: recv got EOF from 127.31.156.65:34791 (error 108)
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":113,"threads_started":1}
Aug 12 07:08:50 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:50 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:50.952857 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:50.945764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:56995)
0812 07:08:50.946112 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:50.946129 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:50.946291 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:50.946588 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:50.946591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:50.946846 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:50.946948 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:50.946953 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.947832 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.947835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:50.948691 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:50.948705 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.949359 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.949362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:50.949449 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:50.950135 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:50.950148 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:50.952638 (+ 2490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":106,"threads_started":1}
W20260812 07:08:50.952967 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:50.945852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:56995 (local address 127.31.156.126:33891)
0812 07:08:50.946081 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:50.946086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:50.946245 (+ 159us) server_negotiation.cc:407] Connection header received
0812 07:08:50.946411 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:50.946419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:50.946466 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:50.946563 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:50.947072 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.947667 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.948845 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:50.949101 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:50.949214 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:50.952828 (+ 3614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:56995: BlockingRecv error: recv got EOF from 127.31.156.66:56995 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 12 07:08:51 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:51 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:51.192731 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:51.186112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:45285)
0812 07:08:51.186502 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:51.186521 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:51.186770 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:51.187083 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:51.187086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:51.187325 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:51.187428 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:51.187434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.188327 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.188333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:51.188910 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:51.188919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.189404 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.189408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:51.189498 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:51.190102 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:51.190115 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:51.192587 (+ 2472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":103,"threads_started":1}
W20260812 07:08:51.193024 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:51.186421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:45285 (local address 127.31.156.126:33891)
0812 07:08:51.186591 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:51.186596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:51.186614 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:08:51.186897 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:51.186900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:51.186962 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:51.187056 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:51.187559 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.188200 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.189037 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.189273 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.189383 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:51.192750 (+ 3367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:45285: BlockingRecv error: recv got EOF from 127.31.156.67:45285 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 07:08:51 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:51 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:51.755191 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:51.748485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:57793)
0812 07:08:51.748783 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:51.748796 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:51.748902 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:51.749179 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:51.749185 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:51.749432 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:51.749539 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:51.749544 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.750462 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.750465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:51.751195 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:51.751208 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.751886 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.751890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:51.751965 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:51.752577 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:51.752600 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:51.755053 (+ 2453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":100,"threads_started":1}
W20260812 07:08:51.755419 10685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:51.748596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:57793 (local address 127.31.156.126:33891)
0812 07:08:51.748904 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:51.748908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:51.748927 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:08:51.748975 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:51.748978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:51.749024 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:51.749126 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:51.749672 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.750312 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.751323 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.751697 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.751817 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:51.755236 (+ 3419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:57793: BlockingRecv error: recv got EOF from 127.31.156.65:57793 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":119,"threads_started":1}
Aug 12 07:08:51 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:51 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:51.960114 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:51.953916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:38167)
0812 07:08:51.954206 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:51.954221 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:51.954340 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:51.954653 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:51.954681 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:51.954890 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:51.954999 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:51.955006 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.955923 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.955925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:51.956545 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:51.956555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.957075 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.957078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:51.957152 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:51.957805 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:51.957821 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:51.959984 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":99,"threads_started":1}
W20260812 07:08:51.960426 10685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:51.953932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:38167 (local address 127.31.156.126:33891)
0812 07:08:51.954113 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:51.954117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:51.954328 (+ 211us) server_negotiation.cc:407] Connection header received
0812 07:08:51.954458 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:51.954461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:51.954516 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:51.954611 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:51.955119 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.955787 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.956674 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:51.956952 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:51.957049 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:51.960196 (+ 3147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:38167: BlockingRecv error: recv got EOF from 127.31.156.66:38167 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 07:08:52 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:52 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:52.201148 10687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:52.194183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:39193)
0812 07:08:52.194620 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:52.194635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:52.194857 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:52.195179 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:52.195182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:52.195453 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:52.195566 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:52.195574 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.196484 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.196490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:52.197192 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:52.197202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.197745 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.197749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:52.197868 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:52.198490 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:52.198513 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:52.200994 (+ 2481us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":173,"threads_started":1}
W20260812 07:08:52.201376 10685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:52.194293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:39193 (local address 127.31.156.126:33891)
0812 07:08:52.194517 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:52.194522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:52.194779 (+ 257us) server_negotiation.cc:407] Connection header received
0812 07:08:52.194980 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:52.194983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:52.195042 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:52.195149 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:52.195710 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.196337 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.197337 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.197599 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.197717 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:52.201176 (+ 3459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:39193: BlockingRecv error: recv got EOF from 127.31.156.67:39193 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 12 07:08:52 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:52 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:52.762802 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:52.756164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:57825)
0812 07:08:52.756534 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:52.756550 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:52.756660 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:52.757004 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:52.757013 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:52.757226 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:52.757336 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:52.757341 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.758205 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.758208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:52.758908 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:52.758919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.759493 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.759496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:52.759569 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:52.760151 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:52.760165 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:52.762584 (+ 2419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":102,"threads_started":1}
W20260812 07:08:52.763083 10689 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:52.756313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:57825 (local address 127.31.156.126:33891)
0812 07:08:52.756606 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:52.756610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:52.756629 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:08:52.756797 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:52.756800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:52.756854 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:52.756948 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:52.757482 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.758066 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.759052 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.759317 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.759461 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:52.762915 (+ 3454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:57825: BlockingRecv error: recv got EOF from 127.31.156.65:57825 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":71,"threads_started":1}
Aug 12 07:08:52 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:52 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:52.968150 10690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:52.961148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:46953)
0812 07:08:52.961635 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:52.961648 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:52.961752 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:52.962048 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:52.962051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:52.962322 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:52.962432 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:52.962438 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.963476 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.963479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:52.964082 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:52.964093 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.964747 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.964750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:52.964826 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:52.965544 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:52.965565 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:52.968005 (+ 2440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":189,"threads_started":1}
W20260812 07:08:52.968457 10689 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:52.961420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:46953 (local address 127.31.156.126:33891)
0812 07:08:52.961592 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:52.961596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:52.961715 (+ 119us) server_negotiation.cc:407] Connection header received
0812 07:08:52.961838 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:52.961840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:52.961890 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:52.961973 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:52.962588 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.963332 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.964341 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:52.964602 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:52.964716 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:52.968212 (+ 3496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:46953: BlockingRecv error: recv got EOF from 127.31.156.66:46953 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 07:08:53 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:53 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:53.209812 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:53.202180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:33251)
0812 07:08:53.202577 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:53.202595 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:53.202769 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:53.203063 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:53.203066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:53.203428 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:53.203577 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:53.203584 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.204888 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.204892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:53.205721 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:53.205735 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.206367 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.206371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:53.206448 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:53.207446 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:53.207461 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:53.209666 (+ 2205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":107,"threads_started":1}
W20260812 07:08:53.210116 10689 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:53.202239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:33251 (local address 127.31.156.126:33891)
0812 07:08:53.202400 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:53.202405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:53.202700 (+ 295us) server_negotiation.cc:407] Connection header received
0812 07:08:53.202853 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:53.202855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:53.202905 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:53.202974 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:53.203735 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.204497 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.205898 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.206140 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.206252 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:53.209968 (+ 3716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:33251: BlockingRecv error: recv got EOF from 127.31.156.67:33251 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 07:08:53 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:53 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:53.776188 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:53.763927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:52451)
0812 07:08:53.764234 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:53.764247 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:53.764346 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:53.764638 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:53.764647 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:53.764895 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:53.765010 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:53.765026 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.765893 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.765897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:53.766543 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:53.766564 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.767421 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.767425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:53.767504 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:53.773250 (+ 5746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:53.773266 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:53.776057 (+ 2791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":107,"threads_started":1}
W20260812 07:08:53.776404 10694 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:53.764025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:52451 (local address 127.31.156.126:33891)
0812 07:08:53.764297 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:53.764301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:53.764316 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:08:53.764419 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:53.764421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:53.764467 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:53.764552 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:53.765172 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.765758 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.766816 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.767090 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.767196 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:53.776228 (+ 9032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:52451: BlockingRecv error: recv got EOF from 127.31.156.65:52451 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":99,"threads_started":1}
Aug 12 07:08:53 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:53 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:53.977325 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:53.969224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:52219)
0812 07:08:53.969811 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:53.969831 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:53.969956 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:53.970247 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:53.970249 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:53.970467 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:53.970574 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:53.970579 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.971690 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.971695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:53.972401 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:53.972420 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.972962 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.972966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:53.973051 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:53.973924 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:53.973938 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:53.977044 (+ 3106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":105,"threads_started":1}
W20260812 07:08:53.977334 10694 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:53.969597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:52219 (local address 127.31.156.126:33891)
0812 07:08:53.969804 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:53.969810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:53.969907 (+ 97us) server_negotiation.cc:407] Connection header received
0812 07:08:53.970037 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:53.970041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:53.970092 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:53.970196 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:53.970835 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.971539 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.972560 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:53.972800 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:53.972900 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:53.977214 (+ 4314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:52219: BlockingRecv error: recv got EOF from 127.31.156.66:52219 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 07:08:54 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:54 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:54.218108 10696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:54.211097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:33337)
0812 07:08:54.211416 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:54.211433 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:54.211572 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:54.211889 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:54.211892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:54.212202 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:54.212350 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:54.212359 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.213213 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.213218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:54.213855 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:54.213864 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.214443 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.214448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:54.214549 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:54.215382 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:54.215396 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:54.217956 (+ 2560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":94,"threads_started":1}
W20260812 07:08:54.218374 10694 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:54.211163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:33337 (local address 127.31.156.126:33891)
0812 07:08:54.211309 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:54.211314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:54.211499 (+ 185us) server_negotiation.cc:407] Connection header received
0812 07:08:54.211695 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:54.211699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:54.211764 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:54.211858 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:54.212484 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.213085 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.214003 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.214270 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.214390 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:54.218160 (+ 3770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:33337: BlockingRecv error: recv got EOF from 127.31.156.67:33337 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 07:08:54 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:54 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:54.784904 10697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:54.777255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:37435)
0812 07:08:54.777844 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:54.777861 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:54.777993 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:54.778466 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:54.778468 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:54.778764 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:54.778879 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:54.778886 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.779962 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.779966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:54.780711 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:54.780725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.781364 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.781367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:54.781454 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:54.782078 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:54.782093 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:54.784577 (+ 2484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":285,"threads_started":1}
W20260812 07:08:54.784938 10698 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:54.777401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:37435 (local address 127.31.156.126:33891)
0812 07:08:54.777799 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:54.777803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:54.777955 (+ 152us) server_negotiation.cc:407] Connection header received
0812 07:08:54.778085 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:54.778087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:54.778137 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:54.778289 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:54.779028 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.779815 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.780863 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.781124 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.781364 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:54.784789 (+ 3425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:37435: BlockingRecv error: recv got EOF from 127.31.156.65:37435 (error 108)
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":97,"threads_started":1}
Aug 12 07:08:54 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:54 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:54.984853 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:54.978232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:57455)
0812 07:08:54.978538 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:54.978554 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:54.978705 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:54.978968 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:54.978971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:54.979175 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:54.979285 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:54.979291 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.980341 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.980345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:54.980999 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:54.981011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.981604 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.981609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:54.981711 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:54.982445 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:54.982459 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:54.984670 (+ 2211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":118,"threads_started":1}
W20260812 07:08:54.984970 10698 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:54.978510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:57455 (local address 127.31.156.126:33891)
0812 07:08:54.978685 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:54.978689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:54.978726 (+ 37us) server_negotiation.cc:407] Connection header received
0812 07:08:54.978786 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:54.978789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:54.978840 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:54.978924 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:54.979518 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.980176 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.981174 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:54.981433 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:54.981536 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:54.984847 (+ 3311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:57455: BlockingRecv error: recv got EOF from 127.31.156.66:57455 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 07:08:55 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:55 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.67@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:55.226341 10701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:55.219122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.67:54957)
0812 07:08:55.219539 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:55.219554 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:55.219672 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:55.220104 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:55.220108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:55.220407 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:55.220519 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:55.220525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.221439 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.221442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:55.222093 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:55.222197 (+ 104us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.222923 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.222927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:55.223028 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:55.223684 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:55.223697 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:55.226114 (+ 2417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":105,"threads_started":1}
W20260812 07:08:55.226384 10698 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:55.219216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:54957 (local address 127.31.156.126:33891)
0812 07:08:55.219396 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:55.219401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:55.219671 (+ 270us) server_negotiation.cc:407] Connection header received
0812 07:08:55.219749 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:55.219752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:55.219804 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:55.219885 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:55.220724 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.221307 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.222398 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.222727 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.222842 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:55.226262 (+ 3420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.67:54957: BlockingRecv error: recv got EOF from 127.31.156.67:54957 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 07:08:55 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:55 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.65@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:55.793735 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:55.786227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.65:53765)
0812 07:08:55.786626 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:55.786641 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:55.786817 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:55.787121 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:55.787124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:55.787411 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:55.787528 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:55.787534 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.788520 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.788524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:55.789185 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:55.789197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.789935 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.789940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:55.790056 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:55.791007 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:55.791028 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:55.793549 (+ 2521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":154,"threads_started":1}
W20260812 07:08:55.793983 10703 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:55.786460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:53765 (local address 127.31.156.126:33891)
0812 07:08:55.786825 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:55.786829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:55.786871 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:08:55.786917 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:55.786919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:55.786963 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:55.787050 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:55.787730 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.788313 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.789395 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.789652 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.789854 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:55.793812 (+ 3958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.65:53765: BlockingRecv error: recv got EOF from 127.31.156.65:53765 (error 108)
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":77,"threads_started":1}
Aug 12 07:08:55 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:08:55 dist-test-slave-csg5 krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.156.66@KRBTEST.COM for oryx/127.31.156.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:08:55.993361 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:55.986169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33891 (local address 127.31.156.66:54615)
0812 07:08:55.986470 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:55.986485 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:55.986595 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:55.986933 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:55.986935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:55.987159 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:55.987340 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:55.987347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.988321 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.988333 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:55.988959 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:55.988969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.989561 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.989565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:55.989640 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:55.990365 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:55.990382 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:55.993216 (+ 2834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.156.126:33891: Server oryx/127.31.156.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":102,"threads_started":1}
W20260812 07:08:55.993701 10703 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:55.986304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:54615 (local address 127.31.156.126:33891)
0812 07:08:55.986467 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:55.986471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:55.986557 (+ 86us) server_negotiation.cc:407] Connection header received
0812 07:08:55.986753 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:55.986755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:55.986812 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:55.986882 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:55.987503 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.988110 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.989106 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:55.989398 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:55.989507 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:55.993567 (+ 4060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.156.66:54615: BlockingRecv error: recv got EOF from 127.31.156.66:54615 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:08:56.089908 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 10141
I20260812 07:08:56.102378 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 10318
I20260812 07:08:56.113590 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 10472
I20260812 07:08:56.125026 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 10058
2026-08-12T07:08:56Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18684 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T07:08:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:56Z Disabled control of system clock
I20260812 07:08:56.166399 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:43949
--webserver_interface=127.31.156.126
--webserver_port=0
--builtin_ntp_servers=127.31.156.84:40153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:43949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 07:08:56.304792 10708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:56.305146 10708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:56.305188 10708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:56.309731 10708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:56.309836 10708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:56.309854 10708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:56.309871 10708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:56.309887 10708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:56.316100 10708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:40153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:43949
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:43949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:56.317382 10708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:56.318950 10708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:56.329871 10717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:56.329885 10714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:56.329885 10713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:56.329949 10708 server_base.cc:1034] running on GCE node
I20260812 07:08:56.331074 10708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:56.332408 10708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:56.333637 10708 hybrid_clock.cc:648] HybridClock initialized: now 1786518536333602 us; error 58 us; skew 500 ppm
I20260812 07:08:56.336102 10708 webserver.cc:460] Webserver started at http://127.31.156.126:43545/ using document root <none> and password file <none>
I20260812 07:08:56.336822 10708 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:56.336930 10708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:56.337204 10708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:56.339784 10708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "e6580332c055407295bd6426a32aae93"
format_stamp: "Formatted at 2026-08-12 07:08:56 on dist-test-slave-csg5"
server_key: "d60e2cc982597dc92a08a7b1fe1e47a2"
server_key_iv: "e6ca8cb0aba890c05c557d026ba222a8"
server_key_version: "encryptionkey@0"
I20260812 07:08:56.340449 10708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "e6580332c055407295bd6426a32aae93"
format_stamp: "Formatted at 2026-08-12 07:08:56 on dist-test-slave-csg5"
server_key: "d60e2cc982597dc92a08a7b1fe1e47a2"
server_key_iv: "e6ca8cb0aba890c05c557d026ba222a8"
server_key_version: "encryptionkey@0"
I20260812 07:08:56.345142 10708 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:56.348595 10723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:56.350584 10708 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 07:08:56.350833 10708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "e6580332c055407295bd6426a32aae93"
format_stamp: "Formatted at 2026-08-12 07:08:56 on dist-test-slave-csg5"
server_key: "d60e2cc982597dc92a08a7b1fe1e47a2"
server_key_iv: "e6ca8cb0aba890c05c557d026ba222a8"
server_key_version: "encryptionkey@0"
I20260812 07:08:56.351083 10708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:56.403509 10708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:56.404348 10708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:56.404531 10708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:56.413800 10708 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:43949
I20260812 07:08:56.413805 10791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:43949 every 8 connection(s)
I20260812 07:08:56.415088 10708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:56.416697 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 10708
I20260812 07:08:56.416839 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:56.417171 32369 external_mini_cluster.cc:1428] Setting key fc2406e3a87357e300228d9bd4346d88
I20260812 07:08:56.419713 10794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:56.426092 10794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93: Bootstrap starting.
I20260812 07:08:56.427884 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:56.418856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41760 (local address 127.31.156.126:43949)
0812 07:08:56.419713 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:56.419736 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:56.419819 (+ 83us) server_negotiation.cc:407] Connection header received
0812 07:08:56.420662 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:56.420683 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:56.421136 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:56.421563 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:56.422050 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:56.423182 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:56.424149 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:56.424673 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:56.424862 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:56.425900 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:56.425905 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:56.425919 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:56.426413 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:56.426477 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:56.426499 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:56.426607 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:56.426802 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 07:08:56.427093 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":218,"threads_started":1}
I20260812 07:08:56.428903 10794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:56.430039 10794 log.cc:824] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:56.432785 10794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93: No bootstrap required, opened a new log
I20260812 07:08:56.437170 10794 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6580332c055407295bd6426a32aae93" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43949 } }
I20260812 07:08:56.437496 10794 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:56.437536 10794 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6580332c055407295bd6426a32aae93, State: Initialized, Role: FOLLOWER
I20260812 07:08:56.438153 10794 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6580332c055407295bd6426a32aae93" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43949 } }
I20260812 07:08:56.438313 10794 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:56.438364 10794 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:56.438566 10794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:56.439996 10794 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6580332c055407295bd6426a32aae93" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43949 } }
I20260812 07:08:56.440548 10794 leader_election.cc:302] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6580332c055407295bd6426a32aae93; no voters:
I20260812 07:08:56.440980 10794 leader_election.cc:288] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:56.441129 10799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:56.441504 10799 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [term 1 LEADER]: Becoming Leader. State: Replica: e6580332c055407295bd6426a32aae93, State: Running, Role: LEADER
I20260812 07:08:56.441885 10799 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6580332c055407295bd6426a32aae93" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43949 } }
I20260812 07:08:56.442485 10794 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:56.444123 10800 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6580332c055407295bd6426a32aae93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6580332c055407295bd6426a32aae93" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43949 } } }
I20260812 07:08:56.444283 10800 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:56.444157 10801 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6580332c055407295bd6426a32aae93. Latest consensus state: current_term: 1 leader_uuid: "e6580332c055407295bd6426a32aae93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6580332c055407295bd6426a32aae93" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 43949 } } }
I20260812 07:08:56.444366 10801 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:56.444799 10811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:56.448606 10811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:56.455543 10811 catalog_manager.cc:1347] Generated new cluster ID: acad756abea94633b654a7fa503b11cc
I20260812 07:08:56.455822 10811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:56.479962 10811 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:56.481384 10811 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:56.494915 10811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6580332c055407295bd6426a32aae93: Generated new TSK 0
I20260812 07:08:56.496012 10811 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:56.503710 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:43949
--builtin_ntp_servers=127.31.156.84:40153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 07:08:56.637789 10822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:56.638181 10822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:56.638266 10822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:56.642691 10822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:56.642798 10822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:56.642890 10822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:56.648361 10822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:40153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:43949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:56.649767 10822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:56.651346 10822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:56.663278 10827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:56.663384 10830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:56.663378 10822 server_base.cc:1034] running on GCE node
W20260812 07:08:56.663551 10828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:56.664057 10822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:56.664877 10822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:56.666119 10822 hybrid_clock.cc:648] HybridClock initialized: now 1786518536666106 us; error 40 us; skew 500 ppm
I20260812 07:08:56.668900 10822 webserver.cc:460] Webserver started at http://127.31.156.65:43339/ using document root <none> and password file <none>
I20260812 07:08:56.669667 10822 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:56.669795 10822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:56.670089 10822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:56.672730 10822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "7939164c95494afeb952fb6d8d4912b9"
format_stamp: "Formatted at 2026-08-12 07:08:56 on dist-test-slave-csg5"
server_key: "2940f5ca8a41e95c4d87517cf81b32b5"
server_key_iv: "6923216a2ab09f8d0c88480e4bc1d9d0"
server_key_version: "encryptionkey@0"
I20260812 07:08:56.673545 10822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "7939164c95494afeb952fb6d8d4912b9"
format_stamp: "Formatted at 2026-08-12 07:08:56 on dist-test-slave-csg5"
server_key: "2940f5ca8a41e95c4d87517cf81b32b5"
server_key_iv: "6923216a2ab09f8d0c88480e4bc1d9d0"
server_key_version: "encryptionkey@0"
I20260812 07:08:56.678819 10822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260812 07:08:56.682441 10836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:56.684559 10822 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:08:56.684760 10822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "7939164c95494afeb952fb6d8d4912b9"
format_stamp: "Formatted at 2026-08-12 07:08:56 on dist-test-slave-csg5"
server_key: "2940f5ca8a41e95c4d87517cf81b32b5"
server_key_iv: "6923216a2ab09f8d0c88480e4bc1d9d0"
server_key_version: "encryptionkey@0"
I20260812 07:08:56.684928 10822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:56.714515 10822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:56.715394 10822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:56.715626 10822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:56.716323 10822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:56.717396 10822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:56.717469 10822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:56.717542 10822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:56.717584 10822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:56.729279 10822 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:41099
I20260812 07:08:56.729353 10969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:41099 every 8 connection(s)
I20260812 07:08:56.730432 10822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:56.734437 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 10822
I20260812 07:08:56.734589 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:08:56.734980 32369 external_mini_cluster.cc:1428] Setting key 036adfe0a06bc37667ad7b56d231189f
I20260812 07:08:56.737563 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:43949
--builtin_ntp_servers=127.31.156.84:40153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 07:08:56.743170 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:56.732674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:59683 (local address 127.31.156.126:43949)
0812 07:08:56.732858 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:56.732862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:56.734215 (+ 1353us) server_negotiation.cc:407] Connection header received
0812 07:08:56.736366 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:56.736370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:56.736437 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:56.736521 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:56.738031 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:56.738785 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:56.740225 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:56.740502 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:56.740650 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:56.742415 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:56.742419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:56.742421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:56.742438 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:56.742460 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:56.742468 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:56.742553 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:56.742863 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 07:08:56.742970 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:08:56.744345 10972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:56.733191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:43949 (local address 127.31.156.65:59683)
0812 07:08:56.734018 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:56.734068 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:56.736093 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:56.736975 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:56.736984 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:56.737090 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:56.737744 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:56.737760 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:56.738968 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:56.738971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:56.740051 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:56.740063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:56.740708 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:56.740713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:56.740919 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:56.741879 (+ 960us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:56.741881 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:56.741934 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:56.742019 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:56.742575 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:56.742588 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:56.742597 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 07:08:56.742872 (+ 275us) client_negotiation.cc:240] Negotiation successful
0812 07:08:56.743260 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":135,"threads_started":1}
I20260812 07:08:56.745872 10970 heartbeater.cc:347] Connected to a master server at 127.31.156.126:43949
I20260812 07:08:56.746198 10970 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:56.747051 10970 heartbeater.cc:511] Master 127.31.156.126:43949 requested a full tablet report, sending...
I20260812 07:08:56.749163 10747 ts_manager.cc:194] Registered new tserver with Master: 7939164c95494afeb952fb6d8d4912b9 (127.31.156.65:41099)
I20260812 07:08:56.751076 10747 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.65:59683
W20260812 07:08:56.864934 10973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:56.865293 10973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:56.865382 10973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:56.869752 10973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:56.869835 10973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:56.869894 10973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:56.875144 10973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:40153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:43949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:56.876473 10973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:56.878083 10973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:56.890782 10979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:56.890776 10982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:56.890776 10978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:56.891619 10973 server_base.cc:1034] running on GCE node
I20260812 07:08:56.892125 10973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:56.892831 10973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:56.894016 10973 hybrid_clock.cc:648] HybridClock initialized: now 1786518536893968 us; error 63 us; skew 500 ppm
I20260812 07:08:56.896811 10973 webserver.cc:460] Webserver started at http://127.31.156.66:40825/ using document root <none> and password file <none>
I20260812 07:08:56.897488 10973 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:56.897560 10973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:56.897816 10973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:56.900398 10973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "7fb9ca58a83742d8be13812a1c480528"
format_stamp: "Formatted at 2026-08-12 07:08:56 on dist-test-slave-csg5"
server_key: "8c946bb9ee9be6d17a342e339a67b091"
server_key_iv: "3c0f38cc0987d6e4592d8d6728381cb8"
server_key_version: "encryptionkey@0"
I20260812 07:08:56.901436 10973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "7fb9ca58a83742d8be13812a1c480528"
format_stamp: "Formatted at 2026-08-12 07:08:56 on dist-test-slave-csg5"
server_key: "8c946bb9ee9be6d17a342e339a67b091"
server_key_iv: "3c0f38cc0987d6e4592d8d6728381cb8"
server_key_version: "encryptionkey@0"
I20260812 07:08:56.906320 10973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260812 07:08:56.909826 10988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:56.911650 10973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:56.911854 10973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "7fb9ca58a83742d8be13812a1c480528"
format_stamp: "Formatted at 2026-08-12 07:08:56 on dist-test-slave-csg5"
server_key: "8c946bb9ee9be6d17a342e339a67b091"
server_key_iv: "3c0f38cc0987d6e4592d8d6728381cb8"
server_key_version: "encryptionkey@0"
I20260812 07:08:56.912000 10973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:56.933382 10973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:56.934279 10973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:56.934513 10973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:56.935330 10973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:56.936882 10973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:56.936934 10973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:56.936980 10973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:56.936995 10973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:56.948336 10973 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:43309
I20260812 07:08:56.948364 11129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:43309 every 8 connection(s)
I20260812 07:08:56.949429 10973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:56.957383 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 10973
I20260812 07:08:56.957523 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:08:56.957864 32369 external_mini_cluster.cc:1428] Setting key a6be4193c4b1ccfb501e0419b04d9abb
I20260812 07:08:56.960423 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:43949
--builtin_ntp_servers=127.31.156.84:40153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 07:08:56.962561 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:56.951918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:41527 (local address 127.31.156.126:43949)
0812 07:08:56.952106 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:56.952110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:56.953700 (+ 1590us) server_negotiation.cc:407] Connection header received
0812 07:08:56.955495 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:56.955499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:56.955563 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:56.955672 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:56.957102 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:56.958028 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:56.959957 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:56.960212 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:56.960356 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:56.961915 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:56.961919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:56.961921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:56.961935 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:56.961957 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:56.961966 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:56.962058 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:56.962287 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 07:08:56.962380 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 07:08:56.963637 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:56.952562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:43949 (local address 127.31.156.66:41527)
0812 07:08:56.953494 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:56.953549 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:56.955197 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:56.955917 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:56.955926 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:56.956025 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:56.956698 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:56.956713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:56.958253 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:56.958259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:56.959804 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:56.959819 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:56.960428 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:56.960433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:56.960730 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:56.961546 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:56.961548 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:56.961600 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:56.961688 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:56.962062 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:56.962076 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:56.962085 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 07:08:56.962262 (+ 177us) client_negotiation.cc:240] Negotiation successful
0812 07:08:56.962639 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":185,"threads_started":1}
I20260812 07:08:56.965195 11130 heartbeater.cc:347] Connected to a master server at 127.31.156.126:43949
I20260812 07:08:56.965606 11130 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:56.966457 11130 heartbeater.cc:511] Master 127.31.156.126:43949 requested a full tablet report, sending...
I20260812 07:08:56.968057 10747 ts_manager.cc:194] Registered new tserver with Master: 7fb9ca58a83742d8be13812a1c480528 (127.31.156.66:43309)
I20260812 07:08:56.968837 10747 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.66:41527
W20260812 07:08:57.093341 11133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:57.093695 11133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:57.093741 11133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:57.098315 11133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:57.098405 11133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:57.098465 11133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:57.103766 11133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:40153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:43949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:57.105082 11133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:57.106632 11133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:08:57.118240 11133 server_base.cc:1034] running on GCE node
W20260812 07:08:57.118203 11139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:57.118328 11141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:57.118541 11138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:57.119100 11133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:57.120029 11133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:57.121253 11133 hybrid_clock.cc:648] HybridClock initialized: now 1786518537121233 us; error 40 us; skew 500 ppm
I20260812 07:08:57.123898 11133 webserver.cc:460] Webserver started at http://127.31.156.67:42871/ using document root <none> and password file <none>
I20260812 07:08:57.124614 11133 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:57.124702 11133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:57.124967 11133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:57.127820 11133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "d6c5097932ff4e319e9e6fbafc74af41"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "7d65dd06e44c8cba7b473201980dacda"
server_key_iv: "4f24f1325ca507d01ac0fdff344a3357"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.128571 11133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "d6c5097932ff4e319e9e6fbafc74af41"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "7d65dd06e44c8cba7b473201980dacda"
server_key_iv: "4f24f1325ca507d01ac0fdff344a3357"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.134310 11133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:08:57.137748 11147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:57.139590 11133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:57.139777 11133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "d6c5097932ff4e319e9e6fbafc74af41"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "7d65dd06e44c8cba7b473201980dacda"
server_key_iv: "4f24f1325ca507d01ac0fdff344a3357"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.139935 11133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:57.151847 11133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:57.152678 11133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:57.152943 11133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:57.153646 11133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:57.154839 11133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:57.154896 11133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:57.154940 11133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:57.154955 11133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:57.167909 11133 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:42901
I20260812 07:08:57.167940 11283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:42901 every 8 connection(s)
I20260812 07:08:57.168967 11133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:57.170706 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 11133
I20260812 07:08:57.170850 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:08:57.171388 32369 external_mini_cluster.cc:1428] Setting key 574ff72cce66a690516d182bb22786f0
I20260812 07:08:57.181272 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:57.171624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:50103 (local address 127.31.156.126:43949)
0812 07:08:57.171799 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:57.171807 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:57.173469 (+ 1662us) server_negotiation.cc:407] Connection header received
0812 07:08:57.175402 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:57.175405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:57.175473 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:57.175568 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:57.176666 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:57.177280 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:57.178892 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:57.179165 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:57.179299 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:57.180664 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:57.180669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:57.180672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:57.180689 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:57.180723 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:57.180732 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:57.180823 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:57.181010 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 07:08:57.181088 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:08:57.182405 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:57.172176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:43949 (local address 127.31.156.67:50103)
0812 07:08:57.173313 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:57.173364 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:57.175131 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:57.175790 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:57.175799 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:57.175900 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:57.176495 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:57.176509 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:57.177441 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:57.177444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:57.178732 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:57.178754 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:57.179360 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:57.179365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:57.179571 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:57.180348 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:57.180350 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:57.180399 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:57.180479 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:57.180826 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:57.180841 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:57.180848 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:08:57.181010 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 07:08:57.181354 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":185,"threads_started":1}
I20260812 07:08:57.184119 11284 heartbeater.cc:347] Connected to a master server at 127.31.156.126:43949
I20260812 07:08:57.184408 11284 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:57.185213 11284 heartbeater.cc:511] Master 127.31.156.126:43949 requested a full tablet report, sending...
I20260812 07:08:57.186434 10747 ts_manager.cc:194] Registered new tserver with Master: d6c5097932ff4e319e9e6fbafc74af41 (127.31.156.67:42901)
I20260812 07:08:57.187071 10747 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.67:50103
I20260812 07:08:57.199030 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 07:08:57.202596 10797 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:57.201453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.31.156.126:43949)
0812 07:08:57.201664 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:57.201670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:57.201846 (+ 176us) server_negotiation.cc:407] Connection header received
0812 07:08:57.201989 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:57.201992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:57.202046 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:57.202136 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:57.202425 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41766: BlockingRecv error: recv got EOF from 127.0.0.1:41766 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
I20260812 07:08:57.203198 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 10822
I20260812 07:08:57.224591 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 10973
I20260812 07:08:57.235423 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 11133
I20260812 07:08:57.246351 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 10708
2026-08-12T07:08:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1120 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T07:08:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:57Z Disabled control of system clock
I20260812 07:08:57.288041 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:34579
--webserver_interface=127.31.156.126
--webserver_port=0
--builtin_ntp_servers=127.31.156.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:34579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 07:08:57.424750 11297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:57.425123 11297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:57.425168 11297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:57.429711 11297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:57.429816 11297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:57.429831 11297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:57.429848 11297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:57.429865 11297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:57.435148 11297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:34579
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:34579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:57.436465 11297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:57.438133 11297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:57.447772 11307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:57.447808 11304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:57.447772 11302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:57.448253 11297 server_base.cc:1034] running on GCE node
I20260812 07:08:57.448984 11297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:57.450335 11297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:57.451597 11297 hybrid_clock.cc:648] HybridClock initialized: now 1786518537451529 us; error 109 us; skew 500 ppm
I20260812 07:08:57.453923 11297 webserver.cc:460] Webserver started at http://127.31.156.126:45161/ using document root <none> and password file <none>
I20260812 07:08:57.454785 11297 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:57.454867 11297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:57.455118 11297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:57.457597 11297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "5d5c361f8be044aa9b84f7fbb9c1156a"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "a8aba28837485d99ed4996e56e599787"
server_key_iv: "422c229f458547a771757b65387bd0f3"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.458266 11297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "5d5c361f8be044aa9b84f7fbb9c1156a"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "a8aba28837485d99ed4996e56e599787"
server_key_iv: "422c229f458547a771757b65387bd0f3"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.462970 11297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:08:57.466137 11314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:57.468036 11297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:57.468356 11297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "5d5c361f8be044aa9b84f7fbb9c1156a"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "a8aba28837485d99ed4996e56e599787"
server_key_iv: "422c229f458547a771757b65387bd0f3"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.468513 11297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:57.474009 11297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:57.474870 11297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:57.475129 11297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:57.483654 11297 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:34579
I20260812 07:08:57.483765 11381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:34579 every 8 connection(s)
I20260812 07:08:57.484798 11297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:57.487804 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 11297
I20260812 07:08:57.487921 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:57.488253 32369 external_mini_cluster.cc:1428] Setting key 828188a21d6277b3c763bccf4473bdad
I20260812 07:08:57.488818 11382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:57.496179 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:57.490072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.31.156.126:34579)
0812 07:08:57.490966 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:57.490989 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:57.491041 (+ 52us) server_negotiation.cc:407] Connection header received
0812 07:08:57.492150 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:57.492180 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:57.492787 (+ 607us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:57.493282 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:57.494268 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:57.494275 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:57.494292 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:57.494307 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:57.494384 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:57.494411 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:57.494590 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:57.494784 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 07:08:57.495074 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":137,"threads_started":1}
I20260812 07:08:57.498039 11382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a: Bootstrap starting.
I20260812 07:08:57.500862 11382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:57.501834 11382 log.cc:824] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:57.504603 11382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a: No bootstrap required, opened a new log
I20260812 07:08:57.509043 11382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c361f8be044aa9b84f7fbb9c1156a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34579 } }
I20260812 07:08:57.509407 11382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:57.509516 11382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5c361f8be044aa9b84f7fbb9c1156a, State: Initialized, Role: FOLLOWER
I20260812 07:08:57.510054 11382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c361f8be044aa9b84f7fbb9c1156a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34579 } }
I20260812 07:08:57.510237 11382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:57.510340 11382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:57.510483 11382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:57.511716 11382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c361f8be044aa9b84f7fbb9c1156a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34579 } }
I20260812 07:08:57.512154 11382 leader_election.cc:302] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d5c361f8be044aa9b84f7fbb9c1156a; no voters:
I20260812 07:08:57.512547 11382 leader_election.cc:288] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:57.512727 11387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:57.512998 11387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [term 1 LEADER]: Becoming Leader. State: Replica: 5d5c361f8be044aa9b84f7fbb9c1156a, State: Running, Role: LEADER
I20260812 07:08:57.513442 11387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c361f8be044aa9b84f7fbb9c1156a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34579 } }
I20260812 07:08:57.514129 11382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:57.515380 11389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d5c361f8be044aa9b84f7fbb9c1156a. Latest consensus state: current_term: 1 leader_uuid: "5d5c361f8be044aa9b84f7fbb9c1156a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c361f8be044aa9b84f7fbb9c1156a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34579 } } }
I20260812 07:08:57.515434 11388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d5c361f8be044aa9b84f7fbb9c1156a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c361f8be044aa9b84f7fbb9c1156a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34579 } } }
I20260812 07:08:57.515671 11388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:57.516189 11398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:57.515650 11389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:57.519543 11398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:57.526141 11398 catalog_manager.cc:1347] Generated new cluster ID: fcede07c58164efe808c4e0104d7f843
I20260812 07:08:57.526239 11398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:57.558804 11398 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:57.559278 11398 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:57.571703 11398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d5c361f8be044aa9b84f7fbb9c1156a: Generated new TSK 0
I20260812 07:08:57.574013 11398 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:57.585493 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34579
--builtin_ntp_servers=127.31.156.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 07:08:57.712044 11408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:57.712384 11408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:57.712431 11408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:57.717417 11408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:57.717505 11408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:57.717569 11408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:08:57.722909 11408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:57.724259 11408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:57.725831 11408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:57.738152 11415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:57.738173 11414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:57.738186 11418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:57.739063 11408 server_base.cc:1034] running on GCE node
I20260812 07:08:57.739746 11408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:57.740582 11408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:57.741814 11408 hybrid_clock.cc:648] HybridClock initialized: now 1786518537741780 us; error 51 us; skew 500 ppm
I20260812 07:08:57.744364 11408 webserver.cc:460] Webserver started at http://127.31.156.65:45985/ using document root <none> and password file <none>
I20260812 07:08:57.745080 11408 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:57.745213 11408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:57.745507 11408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:57.748245 11408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "685c17cda831424e9909451b6a8fb85d"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "f70420925b37aedab45aa56cadc23ce6"
server_key_iv: "8979be3ea4b1e2ae81a7c1b9eef5c780"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.748975 11408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "685c17cda831424e9909451b6a8fb85d"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "f70420925b37aedab45aa56cadc23ce6"
server_key_iv: "8979be3ea4b1e2ae81a7c1b9eef5c780"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.754011 11408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260812 07:08:57.757491 11425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:57.759606 11408 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260812 07:08:57.759800 11408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "685c17cda831424e9909451b6a8fb85d"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "f70420925b37aedab45aa56cadc23ce6"
server_key_iv: "8979be3ea4b1e2ae81a7c1b9eef5c780"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.759964 11408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:57.766207 11408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:57.767114 11408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:57.767347 11408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:57.768006 11408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:57.769111 11408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:57.769191 11408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:57.769277 11408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:57.769325 11408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:57.780409 11408 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:38845
I20260812 07:08:57.780474 11564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:38845 every 8 connection(s)
I20260812 07:08:57.781596 11408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:08:57.785058 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 11408
I20260812 07:08:57.785224 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:08:57.785598 32369 external_mini_cluster.cc:1428] Setting key dd2e0ab8711d84f09e708f4687e816cc
I20260812 07:08:57.788213 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34579
--builtin_ntp_servers=127.31.156.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 07:08:57.789870 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:57.783960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:42555 (local address 127.31.156.126:34579)
0812 07:08:57.784133 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:57.784137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:57.785571 (+ 1434us) server_negotiation.cc:407] Connection header received
0812 07:08:57.787481 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:57.787485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:57.787532 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:57.787634 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:57.789181 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:57.789189 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:57.789196 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:57.789218 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:57.789251 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:57.789268 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:57.789405 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:57.789588 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 07:08:57.789647 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:08:57.790998 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:57.784414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:34579 (local address 127.31.156.65:42555)
0812 07:08:57.785352 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:57.785426 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:57.787157 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:57.787879 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:57.787887 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:57.787997 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:57.788834 (+ 837us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:57.788836 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:57.788894 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:57.788988 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:57.789410 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:57.789420 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:57.789426 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:08:57.789605 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 07:08:57.789908 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":182,"threads_started":1}
I20260812 07:08:57.792523 11565 heartbeater.cc:347] Connected to a master server at 127.31.156.126:34579
I20260812 07:08:57.792836 11565 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:57.793345 11565 heartbeater.cc:511] Master 127.31.156.126:34579 requested a full tablet report, sending...
I20260812 07:08:57.795207 11337 ts_manager.cc:194] Registered new tserver with Master: 685c17cda831424e9909451b6a8fb85d (127.31.156.65:38845)
W20260812 07:08:57.917296 11568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:57.917649 11568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:57.917727 11568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:57.922364 11568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:57.922519 11568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:57.922638 11568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:08:57.928134 11568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:57.929422 11568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:57.930996 11568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:57.943425 11576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:57.943483 11568 server_base.cc:1034] running on GCE node
W20260812 07:08:57.943415 11578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:57.943415 11574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:57.944332 11568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:57.945120 11568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:57.946319 11568 hybrid_clock.cc:648] HybridClock initialized: now 1786518537946301 us; error 32 us; skew 500 ppm
I20260812 07:08:57.948892 11568 webserver.cc:460] Webserver started at http://127.31.156.66:41273/ using document root <none> and password file <none>
I20260812 07:08:57.949577 11568 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:57.949653 11568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:57.949909 11568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:57.952343 11568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "70ee2d6c64bf44119cf6c45d4106a68d"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "812e7a0f44ef476de3767419feba56b6"
server_key_iv: "2ba7573cd725ddcdc6da44b901e67bf3"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.953014 11568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "70ee2d6c64bf44119cf6c45d4106a68d"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "812e7a0f44ef476de3767419feba56b6"
server_key_iv: "2ba7573cd725ddcdc6da44b901e67bf3"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.957657 11568 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:08:57.960829 11585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:57.962636 11568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:57.962914 11568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "70ee2d6c64bf44119cf6c45d4106a68d"
format_stamp: "Formatted at 2026-08-12 07:08:57 on dist-test-slave-csg5"
server_key: "812e7a0f44ef476de3767419feba56b6"
server_key_iv: "2ba7573cd725ddcdc6da44b901e67bf3"
server_key_version: "encryptionkey@0"
I20260812 07:08:57.963100 11568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:57.969321 11568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:57.970108 11568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:57.970346 11568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:57.971065 11568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:57.972394 11568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:57.972457 11568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:57.972538 11568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:57.972577 11568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:57.983001 11568 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:36349
I20260812 07:08:57.983063 11723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:36349 every 8 connection(s)
I20260812 07:08:57.984102 11568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:08:57.988101 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 11568
I20260812 07:08:57.988276 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:08:57.988612 32369 external_mini_cluster.cc:1428] Setting key ab0450256ec56d47c95c5e33d4907c9c
I20260812 07:08:57.991137 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34579
--builtin_ntp_servers=127.31.156.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 07:08:57.992038 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:57.986388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:37101 (local address 127.31.156.126:34579)
0812 07:08:57.986607 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:57.986613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:57.988329 (+ 1716us) server_negotiation.cc:407] Connection header received
0812 07:08:57.989956 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:57.989960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:57.990004 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:57.990085 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:57.991544 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:57.991547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:57.991549 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:57.991565 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:57.991587 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:57.991595 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:57.991704 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:57.991877 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 07:08:57.991920 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 07:08:57.993410 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:57.987079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:34579 (local address 127.31.156.66:37101)
0812 07:08:57.988004 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:57.988076 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:57.989711 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:57.990313 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:57.990323 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:57.990445 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:57.991114 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:57.991117 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:57.991178 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:57.991299 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:57.991704 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:57.991717 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:57.991727 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 07:08:57.991942 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 07:08:57.992244 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":175,"threads_started":1}
I20260812 07:08:57.995013 11724 heartbeater.cc:347] Connected to a master server at 127.31.156.126:34579
I20260812 07:08:57.995308 11724 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:57.995911 11724 heartbeater.cc:511] Master 127.31.156.126:34579 requested a full tablet report, sending...
I20260812 07:08:57.997442 11337 ts_manager.cc:194] Registered new tserver with Master: 70ee2d6c64bf44119cf6c45d4106a68d (127.31.156.66:36349)
W20260812 07:08:58.120386 11727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:58.120702 11727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:58.120747 11727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:58.124991 11727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:58.125065 11727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:58.125183 11727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:08:58.130431 11727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:34579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:58.131728 11727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:58.133224 11727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:58.144914 11732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:58.144918 11733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:58.144914 11736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:58.145303 11727 server_base.cc:1034] running on GCE node
I20260812 07:08:58.145922 11727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:58.146744 11727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:58.147967 11727 hybrid_clock.cc:648] HybridClock initialized: now 1786518538147939 us; error 39 us; skew 500 ppm
I20260812 07:08:58.150431 11727 webserver.cc:460] Webserver started at http://127.31.156.67:37393/ using document root <none> and password file <none>
I20260812 07:08:58.151141 11727 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:58.151244 11727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:58.151500 11727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:58.153939 11727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "6da877c3db4b4b3f91b827017b80197c"
format_stamp: "Formatted at 2026-08-12 07:08:58 on dist-test-slave-csg5"
server_key: "c50ee31c6e7c0cb5011de9b0b86686e0"
server_key_iv: "91c8073627eac1262264064af69935f0"
server_key_version: "encryptionkey@0"
I20260812 07:08:58.154589 11727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "6da877c3db4b4b3f91b827017b80197c"
format_stamp: "Formatted at 2026-08-12 07:08:58 on dist-test-slave-csg5"
server_key: "c50ee31c6e7c0cb5011de9b0b86686e0"
server_key_iv: "91c8073627eac1262264064af69935f0"
server_key_version: "encryptionkey@0"
I20260812 07:08:58.158578 11727 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:08:58.161599 11743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:58.163218 11727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:08:58.163467 11727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "6da877c3db4b4b3f91b827017b80197c"
format_stamp: "Formatted at 2026-08-12 07:08:58 on dist-test-slave-csg5"
server_key: "c50ee31c6e7c0cb5011de9b0b86686e0"
server_key_iv: "91c8073627eac1262264064af69935f0"
server_key_version: "encryptionkey@0"
I20260812 07:08:58.163606 11727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:58.169216 11727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:58.169927 11727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:58.170132 11727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:58.170805 11727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:08:58.171823 11727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:08:58.171875 11727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:58.171916 11727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:08:58.171972 11727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:58.181610 11727 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:46439
I20260812 07:08:58.181648 11875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:46439 every 8 connection(s)
I20260812 07:08:58.182616 11727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:08:58.190084 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:58.184836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:56651 (local address 127.31.156.126:34579)
0812 07:08:58.185003 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:58.185007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:58.186366 (+ 1359us) server_negotiation.cc:407] Connection header received
0812 07:08:58.187960 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:58.187964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:58.188017 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:58.188128 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:58.189541 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:58.189545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:58.189550 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:58.189569 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:58.189591 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:58.189600 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:58.189748 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:58.189877 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 07:08:58.189930 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:08:58.190994 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 11727
I20260812 07:08:58.191115 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:08:58.191258 11878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:58.185313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:34579 (local address 127.31.156.67:56651)
0812 07:08:58.186074 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:58.186122 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:58.187685 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:58.188329 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:58.188336 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:58.188432 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:58.189125 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:08:58.189126 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:58.189179 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:08:58.189344 (+ 165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:58.189692 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:58.189703 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:58.189710 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:08:58.189934 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 07:08:58.190217 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":121,"threads_started":1}
I20260812 07:08:58.191495 32369 external_mini_cluster.cc:1428] Setting key ef24c9364456269f2b37c39a924cacca
I20260812 07:08:58.192571 11876 heartbeater.cc:347] Connected to a master server at 127.31.156.126:34579
I20260812 07:08:58.192911 11876 heartbeater.cc:464] Registering TS with master...
I20260812 07:08:58.193389 11876 heartbeater.cc:511] Master 127.31.156.126:34579 requested a full tablet report, sending...
I20260812 07:08:58.194591 11337 ts_manager.cc:194] Registered new tserver with Master: 6da877c3db4b4b3f91b827017b80197c (127.31.156.67:46439)
I20260812 07:08:58.206734 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 07:08:58.210501 11385 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:58.209232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.31.156.126:34579)
0812 07:08:58.209479 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:58.209485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:58.209565 (+ 80us) server_negotiation.cc:407] Connection header received
0812 07:08:58.209835 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:58.209839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:58.209887 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:58.209979 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:58.210343 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60510: BlockingRecv error: recv got EOF from 127.0.0.1:60510 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
I20260812 07:08:58.210877 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 11408
I20260812 07:08:58.222019 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 11568
I20260812 07:08:58.234608 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 11727
I20260812 07:08:58.243808 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 11297
2026-08-12T07:08:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (995 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:58 dist-test-slave-csg5 krb5kdc[11887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:58 dist-test-slave-csg5 krb5kdc[11887](info): set up 2 sockets
Aug 12 07:08:58 dist-test-slave-csg5 krb5kdc[11887](info): commencing operation
krb5kdc: starting...
W20260812 07:08:59.438097 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:08:59 dist-test-slave-csg5 krb5kdc[11887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:08:59.600942 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:36695
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:36695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:59.728371 11902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:59.728780 11902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:59.728866 11902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:59.733186 11902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:59.733315 11902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:59.733352 11902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:59.733395 11902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:59.733457 11902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:59.739192 11902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:36695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:36695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:08:59.740448 11902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:59.741582 11902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:59.751610 11908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:59.751732 11909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:59.751760 11902 server_base.cc:1034] running on GCE node
W20260812 07:08:59.751601 11912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:59.752717 11902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:59.754098 11902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:59.755404 11902 hybrid_clock.cc:648] HybridClock initialized: now 1786518539755382 us; error 38 us; skew 500 ppm
Aug 12 07:08:59 dist-test-slave-csg5 krb5kdc[11887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518539, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:59.763237 11902 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:08:59.764460 11902 webserver.cc:460] Webserver started at http://127.31.156.126:38933/ using document root <none> and password file <none>
I20260812 07:08:59.765098 11902 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:59.765182 11902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:59.765435 11902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:59.768038 11902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "3a691360d3d148d78f2e54d2834377f0"
format_stamp: "Formatted at 2026-08-12 07:08:59 on dist-test-slave-csg5"
server_key: "5910c62999eac4f4da2320a755815ff3"
server_key_iv: "b852d6f6526b9b41444a130a858f498c"
server_key_version: "encryptionkey@0"
I20260812 07:08:59.768777 11902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "3a691360d3d148d78f2e54d2834377f0"
format_stamp: "Formatted at 2026-08-12 07:08:59 on dist-test-slave-csg5"
server_key: "5910c62999eac4f4da2320a755815ff3"
server_key_iv: "b852d6f6526b9b41444a130a858f498c"
server_key_version: "encryptionkey@0"
I20260812 07:08:59.773568 11902 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 07:08:59.777221 11919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:59.778834 11902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:59.779018 11902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "3a691360d3d148d78f2e54d2834377f0"
format_stamp: "Formatted at 2026-08-12 07:08:59 on dist-test-slave-csg5"
server_key: "5910c62999eac4f4da2320a755815ff3"
server_key_iv: "b852d6f6526b9b41444a130a858f498c"
server_key_version: "encryptionkey@0"
I20260812 07:08:59.779286 11902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:59.811360 11902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:59.815863 11902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:59.816130 11902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:59.824440 11902 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:36695
I20260812 07:08:59.824517 11981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:36695 every 8 connection(s)
I20260812 07:08:59.825610 11902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:59.828871 11982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:59.831367 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 11902
I20260812 07:08:59.831568 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:08:59.831954 32369 external_mini_cluster.cc:1428] Setting key 733aec03b3c0eedef0090a8d7fab75d9
I20260812 07:08:59.836354 11982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0: Bootstrap starting.
I20260812 07:08:59.839476 11982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:59.840508 11982 log.cc:824] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:59.843456 11982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0: No bootstrap required, opened a new log
Aug 12 07:08:59 dist-test-slave-csg5 krb5kdc[11887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:08:59.848063 11982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a691360d3d148d78f2e54d2834377f0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36695 } }
I20260812 07:08:59.848387 11982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:59.848493 11982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a691360d3d148d78f2e54d2834377f0, State: Initialized, Role: FOLLOWER
I20260812 07:08:59.849059 11982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a691360d3d148d78f2e54d2834377f0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36695 } }
I20260812 07:08:59.849269 11982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:59.849370 11982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:59.849511 11982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:59.849743 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:59.833747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.31.156.126:36695)
0812 07:08:59.834262 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:59.834283 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:59.834329 (+ 46us) server_negotiation.cc:407] Connection header received
0812 07:08:59.835461 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:59.835492 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:59.836038 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:59.836494 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:59.837420 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:59.839019 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:59.840041 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:59.840802 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:59.841007 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:59.844248 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:59.844255 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:59.844270 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:59.844300 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:59.846143 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:59.846874 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:59.846880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:59.846906 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:59.847034 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:59.847346 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:59.847350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:59.847351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:59.847817 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:59.848068 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:59.848377 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 07:08:59.848745 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":149,"threads_started":1}
I20260812 07:08:59.850857 11982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a691360d3d148d78f2e54d2834377f0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36695 } }
I20260812 07:08:59.851267 11982 leader_election.cc:302] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a691360d3d148d78f2e54d2834377f0; no voters:
I20260812 07:08:59.851706 11982 leader_election.cc:288] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:59.851850 11992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:59.852097 11992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [term 1 LEADER]: Becoming Leader. State: Replica: 3a691360d3d148d78f2e54d2834377f0, State: Running, Role: LEADER
I20260812 07:08:59.852622 11992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a691360d3d148d78f2e54d2834377f0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36695 } }
I20260812 07:08:59.853225 11982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:59.854470 11994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a691360d3d148d78f2e54d2834377f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a691360d3d148d78f2e54d2834377f0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36695 } } }
I20260812 07:08:59.854612 11994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:59.854959 11995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a691360d3d148d78f2e54d2834377f0. Latest consensus state: current_term: 1 leader_uuid: "3a691360d3d148d78f2e54d2834377f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a691360d3d148d78f2e54d2834377f0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36695 } } }
I20260812 07:08:59.855062 12006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:59.855118 11995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:59.858183 12006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:59.864037 12006 catalog_manager.cc:1347] Generated new cluster ID: 29df0b8c6f9144cc9afb1f8fe2286dd8
I20260812 07:08:59.864137 12006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:59.890126 12006 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:59.891603 12006 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:59.899621 12006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0: Generated new TSK 0
I20260812 07:08:59.900738 12006 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:08:59.944445 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36695
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:00.088737 12019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:00.089077 12019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:00.089133 12019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:00.093402 12019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:00.093493 12019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:00.093557 12019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:09:00.099438 12019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:09:00.100627 12019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:00.101718 12019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:00.112694 12024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:00.112763 12027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:00.112756 12025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:00.113369 12019 server_base.cc:1034] running on GCE node
I20260812 07:09:00.113960 12019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:00.114835 12019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:00.116047 12019 hybrid_clock.cc:648] HybridClock initialized: now 1786518540116006 us; error 56 us; skew 500 ppm
Aug 12 07:09:00 dist-test-slave-csg5 krb5kdc[11887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518540, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:00.119524 12019 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:09:00.120754 12019 webserver.cc:460] Webserver started at http://127.31.156.65:35131/ using document root <none> and password file <none>
I20260812 07:09:00.121410 12019 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:00.121511 12019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:00.121780 12019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:00.124126 12019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "1c9f62596ac54238bc7856ec72ceaec1"
format_stamp: "Formatted at 2026-08-12 07:09:00 on dist-test-slave-csg5"
server_key: "ac12684fa53e87e247f01866cc8fc04d"
server_key_iv: "32018240e6ba5523d3b57dfd4ad83046"
server_key_version: "encryptionkey@0"
I20260812 07:09:00.124864 12019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "1c9f62596ac54238bc7856ec72ceaec1"
format_stamp: "Formatted at 2026-08-12 07:09:00 on dist-test-slave-csg5"
server_key: "ac12684fa53e87e247f01866cc8fc04d"
server_key_iv: "32018240e6ba5523d3b57dfd4ad83046"
server_key_version: "encryptionkey@0"
I20260812 07:09:00.129521 12019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:09:00.132644 12035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:00.134147 12019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:09:00.134477 12019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "1c9f62596ac54238bc7856ec72ceaec1"
format_stamp: "Formatted at 2026-08-12 07:09:00 on dist-test-slave-csg5"
server_key: "ac12684fa53e87e247f01866cc8fc04d"
server_key_iv: "32018240e6ba5523d3b57dfd4ad83046"
server_key_version: "encryptionkey@0"
I20260812 07:09:00.134650 12019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:00.161132 12019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:00.165269 12019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:00.165551 12019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:00.166299 12019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:00.167519 12019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:00.167629 12019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:00.167708 12019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:00.167759 12019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:00.179446 12019 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:44059
I20260812 07:09:00.179467 12168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:44059 every 8 connection(s)
I20260812 07:09:00.180576 12019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:09:00.185698 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 12019
I20260812 07:09:00.185856 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:09:00.186165 32369 external_mini_cluster.cc:1428] Setting key 863842658f14adc86dda324ce6a5ea67
Aug 12 07:09:00 dist-test-slave-csg5 krb5kdc[11887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518540, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:09:00.197894 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.182895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:59301 (local address 127.31.156.126:36695)
0812 07:09:00.183079 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:00.183083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:00.184190 (+ 1107us) server_negotiation.cc:407] Connection header received
0812 07:09:00.185786 (+ 1596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:00.185789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:00.185852 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:00.185940 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:00.187401 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.188156 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.189401 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.189657 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.189777 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.192662 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:00.192665 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:00.192667 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:00.192696 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:00.194984 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:00.195882 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:00.195886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:00.195887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:00.195992 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:00.196448 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:00.196452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:00.196454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:00.196700 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:00.196841 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:00.197509 (+ 668us) server_negotiation.cc:299] Negotiation successful
0812 07:09:00.197674 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 07:09:00.199019 12172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.183394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:36695 (local address 127.31.156.65:59301)
0812 07:09:00.184022 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:00.184070 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:00.185550 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:00.186121 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:00.186129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:00.186626 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:00.187246 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.187260 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.188296 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.188300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.189263 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.189275 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.189839 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.189844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.190035 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.190785 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:00.190805 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:00.192434 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:00.195183 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:00.195197 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:00.195223 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:00.195678 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:00.196112 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:00.196115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:00.196118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:00.196317 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:00.196841 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:00.196852 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:00.197195 (+ 343us) client_negotiation.cc:769] Sending connection context
0812 07:09:00.197435 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 07:09:00.197854 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":171,"threads_started":1}
I20260812 07:09:00.200591 12169 heartbeater.cc:347] Connected to a master server at 127.31.156.126:36695
I20260812 07:09:00.200850 12169 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
I20260812 07:09:00.201530 12169 heartbeater.cc:511] Master 127.31.156.126:36695 requested a full tablet report, sending...
I20260812 07:09:00.203524 11938 ts_manager.cc:194] Registered new tserver with Master: 1c9f62596ac54238bc7856ec72ceaec1 (127.31.156.65:44059)
I20260812 07:09:00.204795 11938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:59301
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:09:00.224642 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36695
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:00.364435 12176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:00.364773 12176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:00.364825 12176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:00.369892 12176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:00.369971 12176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:00.370043 12176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:09:00.376677 12176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:09:00.377933 12176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:00.379148 12176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:00.393038 12182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:00.393018 12181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:00.393038 12185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:00.393513 12176 server_base.cc:1034] running on GCE node
I20260812 07:09:00.394237 12176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:00.395087 12176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:00.396306 12176 hybrid_clock.cc:648] HybridClock initialized: now 1786518540396272 us; error 52 us; skew 500 ppm
Aug 12 07:09:00 dist-test-slave-csg5 krb5kdc[11887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518540, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:00.400103 12176 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:09:00.401243 12176 webserver.cc:460] Webserver started at http://127.31.156.66:44207/ using document root <none> and password file <none>
I20260812 07:09:00.401873 12176 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:00.401975 12176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:00.402271 12176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:00.404690 12176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "22c1384708d54432abdc75b1c781e1b4"
format_stamp: "Formatted at 2026-08-12 07:09:00 on dist-test-slave-csg5"
server_key: "fb7cd21a957417d565d5121f1e9fa206"
server_key_iv: "9f49c6d0c31ad2e07cca12e6d81024ef"
server_key_version: "encryptionkey@0"
I20260812 07:09:00.405371 12176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "22c1384708d54432abdc75b1c781e1b4"
format_stamp: "Formatted at 2026-08-12 07:09:00 on dist-test-slave-csg5"
server_key: "fb7cd21a957417d565d5121f1e9fa206"
server_key_iv: "9f49c6d0c31ad2e07cca12e6d81024ef"
server_key_version: "encryptionkey@0"
I20260812 07:09:00.409781 12176 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 07:09:00.413201 12193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:00.414917 12176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:09:00.415102 12176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "22c1384708d54432abdc75b1c781e1b4"
format_stamp: "Formatted at 2026-08-12 07:09:00 on dist-test-slave-csg5"
server_key: "fb7cd21a957417d565d5121f1e9fa206"
server_key_iv: "9f49c6d0c31ad2e07cca12e6d81024ef"
server_key_version: "encryptionkey@0"
I20260812 07:09:00.415251 12176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:00.443369 12176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:00.448669 12176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:00.448973 12176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:00.449715 12176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:00.450870 12176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:00.450953 12176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:00.451045 12176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:00.451107 12176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:00.463810 12176 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:38731
I20260812 07:09:00.463855 12332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:38731 every 8 connection(s)
I20260812 07:09:00.464936 12176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:09:00.475021 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 12176
I20260812 07:09:00.475167 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:09:00.475577 32369 external_mini_cluster.cc:1428] Setting key d156f830bf5e3dff4fff383534b5882c
Aug 12 07:09:00 dist-test-slave-csg5 krb5kdc[11887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518540, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:09:00.483517 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.467505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:39567 (local address 127.31.156.126:36695)
0812 07:09:00.467702 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:00.467706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:00.468856 (+ 1150us) server_negotiation.cc:407] Connection header received
0812 07:09:00.470531 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:00.470536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:00.470617 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:00.470740 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:00.472224 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.472838 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.474223 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.474482 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.474639 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.478409 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:00.478413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:00.478415 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:00.478450 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:00.480597 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:00.481283 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:00.481287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:00.481289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:00.481415 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:00.481829 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:00.481836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:00.481841 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:00.482150 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:00.482270 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:00.482979 (+ 709us) server_negotiation.cc:299] Negotiation successful
0812 07:09:00.483166 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 07:09:00.484647 12336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.467936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:36695 (local address 127.31.156.66:39567)
0812 07:09:00.468606 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:00.468667 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:00.470228 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:00.470964 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:00.470972 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:00.471441 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:00.472003 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.472060 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.473005 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.473008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.474051 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.474062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.474718 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.474726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.474940 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.475931 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:00.475952 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:00.478146 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:00.480767 (+ 2621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:00.480774 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:00.480791 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:00.481158 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:00.481539 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:00.481542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:00.481543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:00.481711 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:00.482280 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:00.482287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:00.482627 (+ 340us) client_negotiation.cc:769] Sending connection context
0812 07:09:00.482893 (+ 266us) client_negotiation.cc:240] Negotiation successful
0812 07:09:00.483215 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":143,"threads_started":1}
I20260812 07:09:00.486497 12333 heartbeater.cc:347] Connected to a master server at 127.31.156.126:36695
I20260812 07:09:00.486832 12333 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:00.487408 12333 heartbeater.cc:511] Master 127.31.156.126:36695 requested a full tablet report, sending...
I20260812 07:09:00.488891 11938 ts_manager.cc:194] Registered new tserver with Master: 22c1384708d54432abdc75b1c781e1b4 (127.31.156.66:38731)
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:09:00.489871 11938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:39567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:09:00.516054 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36695
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:00.650573 12340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:00.651000 12340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:00.651051 12340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:00.656188 12340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:00.656289 12340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:00.656478 12340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:09:00.663836 12340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:36695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:09:00.665125 12340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:00.666280 12340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:00.678007 12346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:00.678045 12348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:00.678010 12345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:00.678714 12340 server_base.cc:1034] running on GCE node
I20260812 07:09:00.679214 12340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:00.679912 12340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:00.681114 12340 hybrid_clock.cc:648] HybridClock initialized: now 1786518540681091 us; error 36 us; skew 500 ppm
Aug 12 07:09:00 dist-test-slave-csg5 krb5kdc[11887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518540, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:00.684895 12340 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:09:00.686322 12340 webserver.cc:460] Webserver started at http://127.31.156.67:45699/ using document root <none> and password file <none>
I20260812 07:09:00.687104 12340 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:00.687193 12340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:00.687444 12340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:00.689970 12340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "4333c942dcd54c1487c27f0d42495c6c"
format_stamp: "Formatted at 2026-08-12 07:09:00 on dist-test-slave-csg5"
server_key: "b50560f60505032b717390687a8d0303"
server_key_iv: "29504819fbd51cacf8b1d314302999ed"
server_key_version: "encryptionkey@0"
I20260812 07:09:00.690748 12340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "4333c942dcd54c1487c27f0d42495c6c"
format_stamp: "Formatted at 2026-08-12 07:09:00 on dist-test-slave-csg5"
server_key: "b50560f60505032b717390687a8d0303"
server_key_iv: "29504819fbd51cacf8b1d314302999ed"
server_key_version: "encryptionkey@0"
I20260812 07:09:00.695628 12340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 07:09:00.699059 12355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:00.700951 12340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:09:00.701359 12340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "4333c942dcd54c1487c27f0d42495c6c"
format_stamp: "Formatted at 2026-08-12 07:09:00 on dist-test-slave-csg5"
server_key: "b50560f60505032b717390687a8d0303"
server_key_iv: "29504819fbd51cacf8b1d314302999ed"
server_key_version: "encryptionkey@0"
I20260812 07:09:00.701565 12340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:00.720046 12340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:00.724556 12340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:00.724818 12340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:00.725517 12340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:00.726461 12340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:00.726537 12340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:00.726612 12340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:00.726699 12340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:00.737561 12340 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:35105
I20260812 07:09:00.737617 12487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:35105 every 8 connection(s)
I20260812 07:09:00.738780 12340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:09:00.747650 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 12340
I20260812 07:09:00.747810 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:09:00.748145 32369 external_mini_cluster.cc:1428] Setting key 9f2f4adc2f2f29015b59ba4250a72929
Aug 12 07:09:00 dist-test-slave-csg5 krb5kdc[11887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518540, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:09:00.757469 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.740963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:52527 (local address 127.31.156.126:36695)
0812 07:09:00.741173 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:00.741177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:00.742138 (+ 961us) server_negotiation.cc:407] Connection header received
0812 07:09:00.743861 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:00.743864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:00.743920 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:00.744000 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:00.745445 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.746026 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.747971 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.748213 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.748475 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.751794 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:00.751797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:00.751800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:00.751831 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:00.753917 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:00.755062 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:00.755066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:00.755068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:00.755311 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:00.755960 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:00.755964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:00.755965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:00.756298 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:00.756402 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:00.757122 (+ 720us) server_negotiation.cc:299] Negotiation successful
0812 07:09:00.757321 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260812 07:09:00.758428 12491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.741392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:36695 (local address 127.31.156.67:52527)
0812 07:09:00.741962 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:00.742010 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:00.743617 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:00.744178 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:00.744184 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:00.744675 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:00.745234 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.745263 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.746179 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.746182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.747745 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.747762 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.748514 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.748518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.748776 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.749575 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:00.749594 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:00.751585 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:00.754215 (+ 2630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:00.754224 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:00.754247 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:00.754644 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:00.755553 (+ 909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:00.755557 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:00.755558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:00.755807 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:00.756467 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:00.756474 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:00.756818 (+ 344us) client_negotiation.cc:769] Sending connection context
0812 07:09:00.757013 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 07:09:00.757358 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":131,"threads_started":1}
I20260812 07:09:00.760234 12489 heartbeater.cc:347] Connected to a master server at 127.31.156.126:36695
I20260812 07:09:00.760547 12489 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:00.761338 12489 heartbeater.cc:511] Master 127.31.156.126:36695 requested a full tablet report, sending...
I20260812 07:09:00.762847 11939 ts_manager.cc:194] Registered new tserver with Master: 4333c942dcd54c1487c27f0d42495c6c (127.31.156.67:35105)
I20260812 07:09:00.763513 11939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:52527
I20260812 07:09:00.775990 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:09:00.787847 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.778380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50478 (local address 127.31.156.126:36695)
0812 07:09:00.778619 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:00.778624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:00.778834 (+ 210us) server_negotiation.cc:407] Connection header received
0812 07:09:00.779027 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:00.779031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:00.779091 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:00.779193 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:00.780014 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.780648 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.781666 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.781922 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.782050 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.783838 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:00.783842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:00.783844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:00.783872 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:00.785571 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:00.786113 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:00.786117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:00.786119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:00.786222 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:00.786560 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:00.786564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:00.786566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:00.787038 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:00.787192 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:00.787495 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 07:09:00.787637 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260812 07:09:00.792182 11939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:09:00.795960 11939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:09:00.812980 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.806485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:38731 (local address 127.0.0.1:44440)
0812 07:09:00.807851 (+ 1366us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:00.807868 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:00.807962 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:00.808628 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:00.808633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:00.808654 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:00.808753 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.808773 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.809867 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.809873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.811185 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.811195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.812326 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.812329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.812448 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.812489 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 07:09:00.812609 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 07:09:00.812753 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1292,"mutex_wait_us":1,"thread_start_us":114,"threads_started":1}
I20260812 07:09:00.813246 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.807784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52220 (local address 127.31.156.65:44059)
0812 07:09:00.808199 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:00.808208 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:00.808231 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:09:00.808304 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:00.808309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:00.808488 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:00.808641 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:00.809112 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.810256 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.811703 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.812309 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.812422 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.812568 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:00.812980 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 07:09:00.813093 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":101,"threads_started":1}
I20260812 07:09:00.813508 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.806529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57838 (local address 127.31.156.67:35105)
0812 07:09:00.807044 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:00.807059 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:00.807802 (+ 743us) server_negotiation.cc:407] Connection header received
0812 07:09:00.807891 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:00.807899 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:00.808128 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:00.808321 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:00.808707 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.809964 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.811590 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.812408 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.812526 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.812660 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:00.812842 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 07:09:00.812955 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":196,"threads_started":1}
I20260812 07:09:00.813522 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.807229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44440 (local address 127.31.156.66:38731)
0812 07:09:00.808199 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:00.808211 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:00.808231 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:09:00.808304 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:00.808321 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:00.808483 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:00.808641 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:00.808885 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.809679 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.811321 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.812171 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.812295 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.812474 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:00.813227 (+ 753us) server_negotiation.cc:299] Negotiation successful
0812 07:09:00.813349 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":171,"threads_started":1}
I20260812 07:09:00.813530 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.807611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:44059 (local address 127.0.0.1:52220)
0812 07:09:00.808003 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:00.808015 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:00.808096 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:00.808788 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:00.808791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:00.808841 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:00.808968 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.808976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.810382 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.810385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.811566 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.811576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.812639 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.812643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.812736 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.812756 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 07:09:00.812826 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 07:09:00.812896 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":107,"threads_started":1}
I20260812 07:09:00.814333 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:00.806355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35105 (local address 127.0.0.1:57838)
0812 07:09:00.807364 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:00.807479 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:00.807618 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:00.808381 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:00.808387 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:00.808410 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:00.808549 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.808555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.810140 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.810144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.811451 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:00.811462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:00.812575 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:00.812579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:00.812672 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:00.812702 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:09:00.812799 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:09:00.812890 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260812 07:09:00.816586 12250 tablet_service.cc:1467] Processing CreateTablet for tablet 7c33a1f103f44fc1916d5bd67ab6ee8d (DEFAULT_TABLE table=test-table [id=6169eb9b9b794b869d6cb724a4db108b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:00.816661 12095 tablet_service.cc:1467] Processing CreateTablet for tablet 7c33a1f103f44fc1916d5bd67ab6ee8d (DEFAULT_TABLE table=test-table [id=6169eb9b9b794b869d6cb724a4db108b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:00.817198 12411 tablet_service.cc:1467] Processing CreateTablet for tablet 7c33a1f103f44fc1916d5bd67ab6ee8d (DEFAULT_TABLE table=test-table [id=6169eb9b9b794b869d6cb724a4db108b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:00.817770 12095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c33a1f103f44fc1916d5bd67ab6ee8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:00.817780 12250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c33a1f103f44fc1916d5bd67ab6ee8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:00.818168 12411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c33a1f103f44fc1916d5bd67ab6ee8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:00.826498 12512 tablet_bootstrap.cc:492] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: Bootstrap starting.
I20260812 07:09:00.826565 12511 tablet_bootstrap.cc:492] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1: Bootstrap starting.
I20260812 07:09:00.826587 12510 tablet_bootstrap.cc:492] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: Bootstrap starting.
I20260812 07:09:00.829037 12511 tablet_bootstrap.cc:654] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:00.829524 12510 tablet_bootstrap.cc:654] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:00.829037 12512 tablet_bootstrap.cc:654] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:00.830024 12511 log.cc:824] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:00.830389 12510 log.cc:824] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:00.831113 12512 log.cc:824] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:00.832427 12511 tablet_bootstrap.cc:492] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1: No bootstrap required, opened a new log
I20260812 07:09:00.832558 12510 tablet_bootstrap.cc:492] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: No bootstrap required, opened a new log
I20260812 07:09:00.832696 12511 ts_tablet_manager.cc:1397] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 07:09:00.832749 12510 ts_tablet_manager.cc:1397] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 07:09:00.836917 12510 raft_consensus.cc:348] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4333c942dcd54c1487c27f0d42495c6c" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35105 } } peers { permanent_uuid: "22c1384708d54432abdc75b1c781e1b4" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 38731 } } peers { permanent_uuid: "1c9f62596ac54238bc7856ec72ceaec1" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44059 } }
I20260812 07:09:00.837185 12512 tablet_bootstrap.cc:492] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: No bootstrap required, opened a new log
I20260812 07:09:00.837221 12510 raft_consensus.cc:374] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:00.837268 12510 raft_consensus.cc:729] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22c1384708d54432abdc75b1c781e1b4, State: Initialized, Role: FOLLOWER
I20260812 07:09:00.837432 12512 ts_tablet_manager.cc:1397] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260812 07:09:00.837823 12511 raft_consensus.cc:348] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4333c942dcd54c1487c27f0d42495c6c" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35105 } } peers { permanent_uuid: "22c1384708d54432abdc75b1c781e1b4" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 38731 } } peers { permanent_uuid: "1c9f62596ac54238bc7856ec72ceaec1" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44059 } }
I20260812 07:09:00.837821 12510 consensus_queue.cc:260] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4333c942dcd54c1487c27f0d42495c6c" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35105 } } peers { permanent_uuid: "22c1384708d54432abdc75b1c781e1b4" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 38731 } } peers { permanent_uuid: "1c9f62596ac54238bc7856ec72ceaec1" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44059 } }
I20260812 07:09:00.838119 12511 raft_consensus.cc:374] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:00.838199 12511 raft_consensus.cc:729] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9f62596ac54238bc7856ec72ceaec1, State: Initialized, Role: FOLLOWER
I20260812 07:09:00.838760 12511 consensus_queue.cc:260] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4333c942dcd54c1487c27f0d42495c6c" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35105 } } peers { permanent_uuid: "22c1384708d54432abdc75b1c781e1b4" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 38731 } } peers { permanent_uuid: "1c9f62596ac54238bc7856ec72ceaec1" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44059 } }
I20260812 07:09:00.839318 12510 ts_tablet_manager.cc:1428] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 07:09:00.839715 12333 heartbeater.cc:503] Master 127.31.156.126:36695 was elected leader, sending a full tablet report...
I20260812 07:09:00.839979 12511 ts_tablet_manager.cc:1428] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 07:09:00.840348 12169 heartbeater.cc:503] Master 127.31.156.126:36695 was elected leader, sending a full tablet report...
I20260812 07:09:00.842621 12512 raft_consensus.cc:348] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4333c942dcd54c1487c27f0d42495c6c" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35105 } } peers { permanent_uuid: "22c1384708d54432abdc75b1c781e1b4" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 38731 } } peers { permanent_uuid: "1c9f62596ac54238bc7856ec72ceaec1" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44059 } }
I20260812 07:09:00.843094 12512 raft_consensus.cc:374] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:00.843173 12512 raft_consensus.cc:729] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4333c942dcd54c1487c27f0d42495c6c, State: Initialized, Role: FOLLOWER
I20260812 07:09:00.843842 12512 consensus_queue.cc:260] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4333c942dcd54c1487c27f0d42495c6c" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35105 } } peers { permanent_uuid: "22c1384708d54432abdc75b1c781e1b4" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 38731 } } peers { permanent_uuid: "1c9f62596ac54238bc7856ec72ceaec1" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44059 } }
I20260812 07:09:00.844614 12489 heartbeater.cc:503] Master 127.31.156.126:36695 was elected leader, sending a full tablet report...
I20260812 07:09:00.845155 12512 ts_tablet_manager.cc:1428] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
W20260812 07:09:00.932245 12170 tablet.cc:2369] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:09:00.966528 12334 tablet.cc:2369] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:09:00.989959 12490 tablet.cc:2369] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:09:01.091811 12517 raft_consensus.cc:482] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:01.092087 12517 raft_consensus.cc:504] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4333c942dcd54c1487c27f0d42495c6c" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35105 } } peers { permanent_uuid: "22c1384708d54432abdc75b1c781e1b4" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 38731 } } peers { permanent_uuid: "1c9f62596ac54238bc7856ec72ceaec1" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44059 } }
I20260812 07:09:01.093719 12517 leader_election.cc:288] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4333c942dcd54c1487c27f0d42495c6c (127.31.156.67:35105), 1c9f62596ac54238bc7856ec72ceaec1 (127.31.156.65:44059)
I20260812 07:09:01.097985 12522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:01.093995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:35105 (local address 127.31.156.66:45229)
0812 07:09:01.094392 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:01.094410 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:01.094588 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:01.094916 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:01.094919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:01.094935 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:01.095073 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:01.095079 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:01.095858 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:01.095863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:01.096731 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:01.096741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:01.097442 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:01.097446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:01.097582 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:01.097612 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:09:01.097705 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 07:09:01.097798 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":3,"thread_start_us":105,"threads_started":1}
I20260812 07:09:01.097961 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:01.094135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:45229 (local address 127.31.156.67:35105)
0812 07:09:01.094291 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:01.094296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:01.094532 (+ 236us) server_negotiation.cc:407] Connection header received
0812 07:09:01.094758 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:01.094761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:01.094809 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:01.094902 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:01.095212 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:01.095735 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:01.096878 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:01.097301 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:01.097442 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:01.097504 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:01.097738 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 07:09:01.097812 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:09:01.098031 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:01.094348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:35981 (local address 127.31.156.65:44059)
0812 07:09:01.094517 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:01.094522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:01.094675 (+ 153us) server_negotiation.cc:407] Connection header received
0812 07:09:01.094880 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:01.094888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:01.094936 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:01.095024 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:01.095331 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:01.095852 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:01.096950 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:01.097392 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:01.097530 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:01.097593 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:01.097835 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 07:09:01.097924 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:09:01.098047 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:01.094081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:44059 (local address 127.31.156.66:35981)
0812 07:09:01.094553 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:01.094567 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:01.094717 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:01.095048 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:01.095051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:01.095068 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:01.095206 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:01.095211 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:01.095969 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:01.095975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:01.096810 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:01.096820 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:01.097530 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:01.097533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:01.097699 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:01.097718 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 07:09:01.097808 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 07:09:01.097886 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":27,"thread_start_us":66,"threads_started":1}
I20260812 07:09:01.098840 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c33a1f103f44fc1916d5bd67ab6ee8d" candidate_uuid: "22c1384708d54432abdc75b1c781e1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9f62596ac54238bc7856ec72ceaec1" is_pre_election: true
I20260812 07:09:01.098870 12434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c33a1f103f44fc1916d5bd67ab6ee8d" candidate_uuid: "22c1384708d54432abdc75b1c781e1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4333c942dcd54c1487c27f0d42495c6c" is_pre_election: true
I20260812 07:09:01.099246 12434 raft_consensus.cc:2454] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22c1384708d54432abdc75b1c781e1b4 in term 0.
I20260812 07:09:01.099267 12117 raft_consensus.cc:2454] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22c1384708d54432abdc75b1c781e1b4 in term 0.
I20260812 07:09:01.099861 12194 leader_election.cc:302] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c9f62596ac54238bc7856ec72ceaec1, 22c1384708d54432abdc75b1c781e1b4, 4333c942dcd54c1487c27f0d42495c6c; no voters:
I20260812 07:09:01.100170 12517 raft_consensus.cc:2781] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:09:01.100284 12517 raft_consensus.cc:482] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:09:01.100350 12517 raft_consensus.cc:3037] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:01.101588 12517 raft_consensus.cc:504] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4333c942dcd54c1487c27f0d42495c6c" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35105 } } peers { permanent_uuid: "22c1384708d54432abdc75b1c781e1b4" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 38731 } } peers { permanent_uuid: "1c9f62596ac54238bc7856ec72ceaec1" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44059 } }
I20260812 07:09:01.102034 12517 leader_election.cc:288] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [CANDIDATE]: Term 1 election: Requested vote from peers 4333c942dcd54c1487c27f0d42495c6c (127.31.156.67:35105), 1c9f62596ac54238bc7856ec72ceaec1 (127.31.156.65:44059)
I20260812 07:09:01.102545 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c33a1f103f44fc1916d5bd67ab6ee8d" candidate_uuid: "22c1384708d54432abdc75b1c781e1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9f62596ac54238bc7856ec72ceaec1"
I20260812 07:09:01.102595 12434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c33a1f103f44fc1916d5bd67ab6ee8d" candidate_uuid: "22c1384708d54432abdc75b1c781e1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4333c942dcd54c1487c27f0d42495c6c"
I20260812 07:09:01.102759 12434 raft_consensus.cc:3037] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:01.102782 12117 raft_consensus.cc:3037] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:01.104229 12434 raft_consensus.cc:2454] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22c1384708d54432abdc75b1c781e1b4 in term 1.
I20260812 07:09:01.104229 12117 raft_consensus.cc:2454] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22c1384708d54432abdc75b1c781e1b4 in term 1.
I20260812 07:09:01.104667 12195 leader_election.cc:302] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c9f62596ac54238bc7856ec72ceaec1, 22c1384708d54432abdc75b1c781e1b4; no voters:
I20260812 07:09:01.104980 12517 raft_consensus.cc:2781] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:01.105314 12517 raft_consensus.cc:686] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 1 LEADER]: Becoming Leader. State: Replica: 22c1384708d54432abdc75b1c781e1b4, State: Running, Role: LEADER
I20260812 07:09:01.105753 12517 consensus_queue.cc:237] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4333c942dcd54c1487c27f0d42495c6c" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35105 } } peers { permanent_uuid: "22c1384708d54432abdc75b1c781e1b4" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 38731 } } peers { permanent_uuid: "1c9f62596ac54238bc7856ec72ceaec1" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44059 } }
I20260812 07:09:01.109603 11939 catalog_manager.cc:5495] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22c1384708d54432abdc75b1c781e1b4 (127.31.156.66). New cstate: current_term: 1 leader_uuid: "22c1384708d54432abdc75b1c781e1b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4333c942dcd54c1487c27f0d42495c6c" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22c1384708d54432abdc75b1c781e1b4" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 38731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c9f62596ac54238bc7856ec72ceaec1" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44059 } health_report { overall_health: UNKNOWN } } }
I20260812 07:09:01.190714 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:01.185226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44444 (local address 127.31.156.66:38731)
0812 07:09:01.185399 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:01.185404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:01.185505 (+ 101us) server_negotiation.cc:407] Connection header received
0812 07:09:01.185699 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:01.185704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:01.185788 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:01.185902 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:01.186191 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:01.187066 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:01.188128 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:01.188506 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:01.188640 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:01.189226 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:01.189964 (+ 738us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:01.190224 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:01.190394 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 07:09:01.190467 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:09:01.197695 12434 raft_consensus.cc:1261] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c [term 1 FOLLOWER]: Refusing update from remote peer 22c1384708d54432abdc75b1c781e1b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:01.197695 12117 raft_consensus.cc:1261] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 1c9f62596ac54238bc7856ec72ceaec1 [term 1 FOLLOWER]: Refusing update from remote peer 22c1384708d54432abdc75b1c781e1b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:01.198588 12523 consensus_queue.cc:1035] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4333c942dcd54c1487c27f0d42495c6c" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 35105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:01.198865 12517 consensus_queue.cc:1035] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c9f62596ac54238bc7856ec72ceaec1" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 44059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:01.206341 12528 mvcc.cc:205] Tried to move back new op lower bound from 7317579944737316864 to 7317579944372944896. Current Snapshot: MvccSnapshot[applied={T|T < 7317579944737316864}]
I20260812 07:09:01.207489 12529 mvcc.cc:205] Tried to move back new op lower bound from 7317579944737316864 to 7317579944372944896. Current Snapshot: MvccSnapshot[applied={T|T < 7317579944737316864}]
I20260812 07:09:01.218082 11939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:01.218334 11939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:01.221897 11939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a691360d3d148d78f2e54d2834377f0: Sending DeleteTablet for 3 replicas of tablet 7c33a1f103f44fc1916d5bd67ab6ee8d
I20260812 07:09:01.223098 12411 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c33a1f103f44fc1916d5bd67ab6ee8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:01 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:57838
I20260812 07:09:01.223078 12250 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c33a1f103f44fc1916d5bd67ab6ee8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:01 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:44440
I20260812 07:09:01.223237 12095 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c33a1f103f44fc1916d5bd67ab6ee8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:01 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:52220
I20260812 07:09:01.223729 12540 tablet_replica.cc:330] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: stopping tablet replica
I20260812 07:09:01.224117 12540 raft_consensus.cc:2229] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:09:01.224272 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 12019
I20260812 07:09:01.229533 12540 raft_consensus.cc:2258] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:01.230703 12541 tablet_replica.cc:330] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: stopping tablet replica
I20260812 07:09:01.231140 12541 raft_consensus.cc:2229] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:01.231552 12540 ts_tablet_manager.cc:1905] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:01.231622 12541 raft_consensus.cc:2258] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:01.233646 12541 ts_tablet_manager.cc:1905] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:01.234828 12540 ts_tablet_manager.cc:1918] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:09:01.234951 12540 log.cc:1196] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/7c33a1f103f44fc1916d5bd67ab6ee8d
I20260812 07:09:01.235304 12540 ts_tablet_manager.cc:1939] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 22c1384708d54432abdc75b1c781e1b4: Deleting consensus metadata
I20260812 07:09:01.237005 11922 catalog_manager.cc:4841] TS 22c1384708d54432abdc75b1c781e1b4 (127.31.156.66:38731): tablet 7c33a1f103f44fc1916d5bd67ab6ee8d (table test-table [id=6169eb9b9b794b869d6cb724a4db108b]) successfully deleted
I20260812 07:09:01.238221 12541 ts_tablet_manager.cc:1918] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:09:01.238449 12541 log.cc:1196] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/7c33a1f103f44fc1916d5bd67ab6ee8d
I20260812 07:09:01.238962 12541 ts_tablet_manager.cc:1939] T 7c33a1f103f44fc1916d5bd67ab6ee8d P 4333c942dcd54c1487c27f0d42495c6c: Deleting consensus metadata
I20260812 07:09:01.240777 11920 catalog_manager.cc:4841] TS 4333c942dcd54c1487c27f0d42495c6c (127.31.156.67:35105): tablet 7c33a1f103f44fc1916d5bd67ab6ee8d (table test-table [id=6169eb9b9b794b869d6cb724a4db108b]) successfully deleted
W20260812 07:09:01.241402 11921 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:44059 (error 108)
I20260812 07:09:01.241710 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 12176
I20260812 07:09:01.245054 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:01.244692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:44059 (local address 127.0.0.1:52224)
0812 07:09:01.244856 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:01.244952 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:44059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 07:09:01.245323 11921 catalog_manager.cc:4584] TS 1c9f62596ac54238bc7856ec72ceaec1 (127.31.156.65:44059): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7c33a1f103f44fc1916d5bd67ab6ee8d: Network error: Client connection negotiation failed: client connection to 127.31.156.65:44059: connect: Connection refused (error 111)
I20260812 07:09:01.251947 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 12340
I20260812 07:09:01.263451 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 11902
2026-08-12T07:09:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (3021 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:09:01 dist-test-slave-csg5 krb5kdc[12547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:09:01 dist-test-slave-csg5 krb5kdc[12547](info): set up 2 sockets
Aug 12 07:09:01 dist-test-slave-csg5 krb5kdc[12547](info): commencing operation
krb5kdc: starting...
W20260812 07:09:02.472939 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:09:02 dist-test-slave-csg5 krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:09:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:09:02Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.156.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.156.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/rangeradmin_127.31.156.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/rangeradmin_127.31.156.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.156.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.156.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/rangerlookup_127.31.156.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/rangerlookup_127.31.156.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.156.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.156.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/HTTP_127.31.156.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/HTTP_127.31.156.84@KRBTEST.COM.keytab.
I20260812 07:09:02.706548 32369 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskAFNoqi/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/postgres -l logfile start
2026-08-12 07:09:06.321 UTC [12581] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 07:09:06.321 UTC [12581] LOG: listening on IPv4 address "127.31.156.84", port 43263
2026-08-12 07:09:06.327 UTC [12581] LOG: listening on Unix socket "/tmp/.s.PGSQL.43263"
2026-08-12 07:09:06.363 UTC [12584] LOG: database system was shut down at 2026-08-12 07:09:03 UTC
2026-08-12 07:09:06.369 UTC [12581] LOG: database system is ready to accept connections
I20260812 07:09:07.424755 32369 mini_postgres.cc:96] Postgres bound to 43263
2026-08-12 07:09:07.438 UTC [12596] FATAL: database "slave" does not exist
127.31.156.84:43263 - accepting connections
I20260812 07:09:07.441005 32369 mini_ranger.cc:161] Starting Ranger...
I20260812 07:09:07.466404 32369 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 07:09:08.227674 32369 mini_ranger.cc:87] Created ranger Postgres database
I20260812 07:09:08.227846 32369 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-admin
2026-08-12 07:09:08,592 [I] DB FLAVOR :POSTGRES
2026-08-12 07:09:08,593 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:09:08,593 [I] Checking connection..
2026-08-12 07:09:08,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 07:09:08,925 [I] Checking connection passed.
2026-08-12 07:09:08,926 [I] --------- Verifying version history table ---------
2026-08-12 07:09:08,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:09:09,208 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:09:09,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:09:09,514 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:09:09,515 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 07:09:09,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 07:09:09.802 UTC [12670] WARNING: there is no transaction in progress
2026-08-12 07:09:09,811 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 07:09:09,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:09:10,121 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 07:09:10,121 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:09:10,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 07:09:10,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 07:09:10,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-csg5.c.gcp-upstream.internal','N') ;"
2026-08-12 07:09:11,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:09:11,350 [I] Table x_portal_user does not exist in database ranger
2026-08-12 07:09:11,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:09:11,636 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 07:09:11,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:09:11,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:09:12,249 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 07:09:12,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 07:09:13.646 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.668 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.690 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.712 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.733 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.761 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.835 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.845 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.866 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.880 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.894 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.913 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.927 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.938 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.950 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:13.960 UTC [12797] WARNING: there is no transaction in progress
2026-08-12 07:09:15,134 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 07:09:15,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 07:09:15,434 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 07:09:15,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:09:15,741 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 07:09:15,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:09:16,037 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 07:09:16,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:09:16,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:09:16,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-csg5.c.gcp-upstream.internal';"
2026-08-12 07:09:16,893 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 07:09:16,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 07:09:17,164 [I] DB_PATCHES have already been applied
I20260812 07:09:17.171111 32369 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-admin:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskAFNoqi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 07:09:17.171188 32369 mini_ranger.cc:193] Using host: 127.31.156.84
I20260812 07:09:17.176270 32369 mini_ranger.cc:227] Ranger admin URL: http://127.31.156.84:53025
Aug 12, 2026 7:09:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:09:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:09:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 7:09:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 7:09:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 7:09:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 7:09:18 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 07:09:18.484966 32369 mini_ranger.cc:160] Time spent starting Ranger: real 11.044s user 0.004s sys 0.012s
Aug 12, 2026 7:09:43 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 7:09:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 7:09:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 7:09:43 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 7:09:44 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 07:09:44.932749 32369 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.156.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.156.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/rangerkms_127.31.156.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/rangerkms_127.31.156.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.156.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.156.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.156.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/HTTP_127.31.156.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.156.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/HTTP_127.31.156.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 07:09:45 dist-test-slave-csg5 krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518585, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 07:09:45.066077 32369 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 07:09:45.080039 32369 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 07:09:46.503042 32369 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 07:09:46.503136 32369 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-kms
I20260812 07:09:46.503260 32369 mini_ranger_kms.cc:227] Using postgres at 127.31.156.84:43263
2026-08-12 07:09:47,387 [I] DB FLAVOR :POSTGRES
2026-08-12 07:09:47,387 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:09:47,387 [I] Checking connection
2026-08-12 07:09:47,387 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 07:09:47,687 [I] connection success
2026-08-12 07:09:47,687 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 07:09:47,687 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 07:09:47,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 07:09:47,993 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 07:09:47,994 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:09:47,994 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 07:09:47,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:43263/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 07:09:48,315 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 07:09:48.924536 32369 mini_ranger_kms.cc:310] Created kms service
I20260812 07:09:49.181571 32369 mini_ranger_kms.cc:326] Created kudu user
I20260812 07:09:49.232188 32369 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 07:09:49 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518585, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:09:49.776029 32369 mini_ranger_kms.cc:384] Added ranger policy
I20260812 07:09:49.776392 32369 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-kms:/tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 07:09:49.776430 32369 mini_ranger_kms.cc:240] Using host: 127.31.156.84
I20260812 07:09:49.781951 32369 mini_ranger_kms.cc:276] Ranger KMS PID: 13008
I20260812 07:09:49.782138 32369 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.156.84:41707
Aug 12, 2026 7:09:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:09:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:09:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 7:09:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 7:09:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 7:09:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 07:09:52 dist-test-slave-csg5 krb5kdc[12547](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518592, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.156.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:52.509148 32369 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.443s user 0.000s sys 0.319s
I20260812 07:09:52.509327 32369 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 07:09:52.509469 32369 mini_ranger_kms.cc:401] 127.31.156.84:41707/kms/v1/keys
Aug 12 07:09:52 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518592, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.156.84@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
Aug 12 07:09:53 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518592, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.156.84@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
Aug 12 07:09:55 dist-test-slave-csg5 krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:09:55.791953 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45773
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:45773
--ranger_config_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:41707/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:55.928102 13067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:55.928439 13067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:55.928483 13067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:55.933789 13067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:09:55.933907 13067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:55.933928 13067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:55.933944 13067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:09:55.933960 13067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:09:55.940623 13067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.156.84:41707/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:45773
--ranger_config_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:09:55.942551 13067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:55.944484 13067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:55.955051 13074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:55.955075 13076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:55.955122 13067 server_base.cc:1034] running on GCE node
W20260812 07:09:55.955500 13080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:55.956348 13067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:55.958057 13067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:55.959273 13067 hybrid_clock.cc:648] HybridClock initialized: now 1786518595959242 us; error 47 us; skew 500 ppm
Aug 12 07:09:55 dist-test-slave-csg5 krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518595, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:55.968052 13067 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:09:55.969322 13067 webserver.cc:460] Webserver started at http://127.31.156.126:41949/ using document root <none> and password file <none>
I20260812 07:09:55.969952 13067 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:55.970016 13067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:55.970274 13067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:09:55 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518595, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:09:56.048396 13067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "0e2bc3480baf4517bdcf9fb903655b9b"
format_stamp: "Formatted at 2026-08-12 07:09:56 on dist-test-slave-csg5"
server_key: "a73b6fd2aceb486b1aee672a9ed29bf6"
server_key_iv: "9f82f3694c6e872cdfb32db403168fc2"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:56.049160 13067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "0e2bc3480baf4517bdcf9fb903655b9b"
format_stamp: "Formatted at 2026-08-12 07:09:56 on dist-test-slave-csg5"
server_key: "a73b6fd2aceb486b1aee672a9ed29bf6"
server_key_iv: "9f82f3694c6e872cdfb32db403168fc2"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:56.054224 13067 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260812 07:09:56.073083 13088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:56.075130 13067 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:09:56.075330 13067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "0e2bc3480baf4517bdcf9fb903655b9b"
format_stamp: "Formatted at 2026-08-12 07:09:56 on dist-test-slave-csg5"
server_key: "a73b6fd2aceb486b1aee672a9ed29bf6"
server_key_iv: "9f82f3694c6e872cdfb32db403168fc2"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:56.075511 13067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:56.099159 13067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:56.103170 13067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:56.103400 13067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:56.112844 13067 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:45773
I20260812 07:09:56.112896 13154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:45773 every 8 connection(s)
I20260812 07:09:56.114151 13067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:09:56.117985 13155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:56.124171 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 13067
I20260812 07:09:56.124329 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:09:56.124646 32369 external_mini_cluster.cc:1428] Setting key 8d1145f886c1624130c44d00b4f8b1dc
I20260812 07:09:56.125494 13155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b: Bootstrap starting.
I20260812 07:09:56.129539 13155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:56.130643 13155 log.cc:824] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:56.133251 13155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b: No bootstrap required, opened a new log
I20260812 07:09:56.139689 13155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e2bc3480baf4517bdcf9fb903655b9b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45773 } }
I20260812 07:09:56.140053 13155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 12 07:09:56 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:09:56.140148 13155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2bc3480baf4517bdcf9fb903655b9b, State: Initialized, Role: FOLLOWER
I20260812 07:09:56.140786 13155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e2bc3480baf4517bdcf9fb903655b9b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45773 } }
I20260812 07:09:56.140993 13155 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:09:56.141120 13155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:09:56.141273 13155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:56.142947 13155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e2bc3480baf4517bdcf9fb903655b9b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45773 } }
I20260812 07:09:56.143427 13155 leader_election.cc:302] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e2bc3480baf4517bdcf9fb903655b9b; no voters:
I20260812 07:09:56.143945 13155 leader_election.cc:288] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:09:56.144052 13160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:56.144450 13160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [term 1 LEADER]: Becoming Leader. State: Replica: 0e2bc3480baf4517bdcf9fb903655b9b, State: Running, Role: LEADER
I20260812 07:09:56.144869 13160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e2bc3480baf4517bdcf9fb903655b9b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45773 } }
I20260812 07:09:56.145607 13155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:09:56.146540 13160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e2bc3480baf4517bdcf9fb903655b9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e2bc3480baf4517bdcf9fb903655b9b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45773 } } }
I20260812 07:09:56.146847 13160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:56.147439 13160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e2bc3480baf4517bdcf9fb903655b9b. Latest consensus state: current_term: 1 leader_uuid: "0e2bc3480baf4517bdcf9fb903655b9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e2bc3480baf4517bdcf9fb903655b9b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45773 } } }
I20260812 07:09:56.147639 13160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:56.148023 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:56.127892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.31.156.126:45773)
0812 07:09:56.128591 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:56.128614 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:56.128668 (+ 54us) server_negotiation.cc:407] Connection header received
0812 07:09:56.129872 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:56.129908 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:56.130432 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:56.130948 (+ 516us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:56.131944 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.133595 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.135019 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.136764 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.139226 (+ 2462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:56.140890 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:56.140898 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:56.140921 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:56.140971 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:56.143430 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:56.144169 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:56.144177 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:56.144185 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:56.144344 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:56.144678 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:56.144681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:56.144682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:56.145291 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:56.146325 (+ 1034us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:56.146496 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 07:09:56.146867 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":202,"threads_started":1}
I20260812 07:09:56.148790 13155 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 07:09:56.150585 13170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:09:56.158185 13170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:09:56.166960 13170 catalog_manager.cc:1347] Generated new cluster ID: 907f7b90958141ff92ef4429ac93a88e
I20260812 07:09:56.167054 13170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:09:56.215425 13170 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:09:56.217077 13170 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:09:56.226184 13170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b: Generated new TSK 0
I20260812 07:09:56.227440 13170 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:09:57.039845 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:41707/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45773
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:57.229727 13207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:57.230192 13207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:57.230264 13207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:57.237432 13207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:57.237557 13207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:57.237666 13207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:09:57.247114 13207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.156.84:41707/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:09:57.248782 13207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:57.250329 13207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:57.266382 13216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:57.266388 13217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:57.266553 13220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:57.268057 13207 server_base.cc:1034] running on GCE node
I20260812 07:09:57.268905 13207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:57.269989 13207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:57.271301 13207 hybrid_clock.cc:648] HybridClock initialized: now 1786518597271270 us; error 59 us; skew 500 ppm
Aug 12 07:09:57 dist-test-slave-csg5 krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:57.275489 13207 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:09:57.277349 13207 webserver.cc:460] Webserver started at http://127.31.156.65:33045/ using document root <none> and password file <none>
I20260812 07:09:57.278120 13207 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:57.278250 13207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:57.278565 13207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:09:57 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:09:57.307447 13207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "5d25ffae1fe8406491a9d4ad659c21ac"
format_stamp: "Formatted at 2026-08-12 07:09:57 on dist-test-slave-csg5"
server_key: "00a089ef34068e8ae2f31a5f6cc2976b"
server_key_iv: "c42d71a023e7cfa136c2584091e2573f"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:57.310045 13207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "5d25ffae1fe8406491a9d4ad659c21ac"
format_stamp: "Formatted at 2026-08-12 07:09:57 on dist-test-slave-csg5"
server_key: "00a089ef34068e8ae2f31a5f6cc2976b"
server_key_iv: "c42d71a023e7cfa136c2584091e2573f"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:57.328593 13207 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.006s sys 0.000s
I20260812 07:09:57.352617 13230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:57.354928 13207 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.004s sys 0.000s
I20260812 07:09:57.355115 13207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "5d25ffae1fe8406491a9d4ad659c21ac"
format_stamp: "Formatted at 2026-08-12 07:09:57 on dist-test-slave-csg5"
server_key: "00a089ef34068e8ae2f31a5f6cc2976b"
server_key_iv: "c42d71a023e7cfa136c2584091e2573f"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:57.355284 13207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:57.414456 13207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:57.419443 13207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:57.419927 13207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:57.422263 13207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:57.424111 13207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:57.424508 13207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 07:09:57.424950 13207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:57.425058 13207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:57.441484 13207 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:40579
I20260812 07:09:57.443830 13207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:09:57.444381 13359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:40579 every 8 connection(s)
I20260812 07:09:57.445468 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 13207
I20260812 07:09:57.445606 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:09:57.445960 32369 external_mini_cluster.cc:1428] Setting key 2a8aa3c51e2ca4a0c8d9307546e8bd41
Aug 12 07:09:57 dist-test-slave-csg5 krb5kdc[12547](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 07:09:57 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:09:57.483829 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:57.450745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:42679 (local address 127.31.156.126:45773)
0812 07:09:57.454615 (+ 3870us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:57.454621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:57.456829 (+ 2208us) server_negotiation.cc:407] Connection header received
0812 07:09:57.458369 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:57.458374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:57.458459 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:57.458564 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:57.461720 (+ 3156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.462707 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.467314 (+ 4607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.467705 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.467837 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:57.471608 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:57.471613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:57.471616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:57.471651 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:57.473829 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.480961 (+ 7132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.481005 (+ 44us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.481007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.481189 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.481942 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.481946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.481948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.482352 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:57.482466 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:57.483420 (+ 954us) server_negotiation.cc:299] Negotiation successful
0812 07:09:57.483611 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3741,"thread_start_us":104,"threads_started":1}
I20260812 07:09:57.486207 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:57.449334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45773 (local address 127.31.156.65:42679)
0812 07:09:57.455731 (+ 6397us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:57.455846 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:57.457799 (+ 1953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:57.459026 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:57.459040 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:57.459685 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:57.461428 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:57.461457 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.462903 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.462906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:57.464400 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:57.464417 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.468156 (+ 3739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.468160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:57.468428 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:57.469277 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:57.469303 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:57.471322 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:57.474078 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:57.474086 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:57.474109 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:57.474777 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:57.481393 (+ 6616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:57.481397 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:57.481400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:57.481726 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:57.482562 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:57.482571 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:57.483044 (+ 473us) client_negotiation.cc:769] Sending connection context
0812 07:09:57.484447 (+ 1403us) client_negotiation.cc:240] Negotiation successful
0812 07:09:57.484905 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6001,"thread_start_us":179,"threads_started":1}
I20260812 07:09:57.488724 13360 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45773
I20260812 07:09:57.489022 13360 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:57.489758 13360 heartbeater.cc:511] Master 127.31.156.126:45773 requested a full tablet report, sending...
I20260812 07:09:57.491993 13114 ts_manager.cc:194] Registered new tserver with Master: 5d25ffae1fe8406491a9d4ad659c21ac (127.31.156.65:40579)
I20260812 07:09:57.493731 13114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:42679
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:09:57.528784 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:41707/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45773
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:57.700407 13367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:57.700966 13367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:57.701120 13367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:57.709126 13367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:57.709327 13367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:57.709494 13367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:09:57.718986 13367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.156.84:41707/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:09:57.720960 13367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:57.722627 13367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:57.738431 13374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:57.739130 13376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:57.740335 13373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:57.740622 13367 server_base.cc:1034] running on GCE node
I20260812 07:09:57.741227 13367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:57.742121 13367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:57.743494 13367 hybrid_clock.cc:648] HybridClock initialized: now 1786518597743396 us; error 114 us; skew 500 ppm
Aug 12 07:09:57 dist-test-slave-csg5 krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:57.747778 13367 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:09:57.749138 13367 webserver.cc:460] Webserver started at http://127.31.156.66:36749/ using document root <none> and password file <none>
I20260812 07:09:57.750011 13367 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:57.750185 13367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:57.750505 13367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:09:57 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:09:57.777136 13367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "b923ffafb90549a98505769cdb42ca73"
format_stamp: "Formatted at 2026-08-12 07:09:57 on dist-test-slave-csg5"
server_key: "d92a3a9bf4d45007da1227977cb4dcbd"
server_key_iv: "c1cf899f3b1b71f1f1a82170ad642a44"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:57.778074 13367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "b923ffafb90549a98505769cdb42ca73"
format_stamp: "Formatted at 2026-08-12 07:09:57 on dist-test-slave-csg5"
server_key: "d92a3a9bf4d45007da1227977cb4dcbd"
server_key_iv: "c1cf899f3b1b71f1f1a82170ad642a44"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:57.784068 13367 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260812 07:09:57.807802 13383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:57.809616 13367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:09:57.809767 13367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "b923ffafb90549a98505769cdb42ca73"
format_stamp: "Formatted at 2026-08-12 07:09:57 on dist-test-slave-csg5"
server_key: "d92a3a9bf4d45007da1227977cb4dcbd"
server_key_iv: "c1cf899f3b1b71f1f1a82170ad642a44"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:57.809917 13367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:57.832441 13367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:57.836560 13367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:57.836844 13367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:57.837661 13367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:57.838879 13367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:57.838999 13367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:57.839125 13367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:57.839195 13367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:57.864782 13367 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:42817
I20260812 07:09:57.866189 13367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:09:57.871174 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 13367
I20260812 07:09:57.871321 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:09:57.871328 13518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:42817 every 8 connection(s)
I20260812 07:09:57.871636 32369 external_mini_cluster.cc:1428] Setting key f30010b1defe7a2df0380dbd569ef697
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
Aug 12 07:09:57 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
I20260812 07:09:57.906087 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:57.873304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:48661 (local address 127.31.156.126:45773)
0812 07:09:57.873446 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:57.873451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:57.881085 (+ 7634us) server_negotiation.cc:407] Connection header received
0812 07:09:57.882956 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:57.882961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:57.883041 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:57.883110 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:57.890403 (+ 7293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.891297 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.894471 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.895004 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.895146 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:57.900293 (+ 5147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:57.900299 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:57.900303 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:57.900345 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:57.902692 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.903651 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.903656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.903659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.903788 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.904317 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.904321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.904323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.904607 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:57.904699 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:57.905680 (+ 981us) server_negotiation.cc:299] Negotiation successful
0812 07:09:57.905887 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:09:57.908453 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:57.873963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45773 (local address 127.31.156.66:48661)
0812 07:09:57.879044 (+ 5081us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:57.880928 (+ 1884us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:57.882407 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:57.883814 (+ 1407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:57.883829 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:57.884294 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:57.890099 (+ 5805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:57.890129 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.891519 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.891523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:57.894277 (+ 2754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:57.894296 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.895427 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.895432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:57.895717 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:57.896738 (+ 1021us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:57.896765 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:57.899954 (+ 3189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:57.902936 (+ 2982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:57.902945 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:57.902973 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:57.903485 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:57.903927 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:57.903930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:57.903933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:57.904173 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:57.904815 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:57.904824 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:57.905308 (+ 484us) client_negotiation.cc:769] Sending connection context
0812 07:09:57.906714 (+ 1406us) client_negotiation.cc:240] Negotiation successful
0812 07:09:57.907163 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4705,"thread_start_us":201,"threads_started":1}
I20260812 07:09:57.911048 13519 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45773
I20260812 07:09:57.911391 13519 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:57.912139 13519 heartbeater.cc:511] Master 127.31.156.126:45773 requested a full tablet report, sending...
I20260812 07:09:57.913828 13114 ts_manager.cc:194] Registered new tserver with Master: b923ffafb90549a98505769cdb42ca73 (127.31.156.66:42817)
I20260812 07:09:57.914716 13114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:48661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:09:57.941411 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:41707/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45773
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:09:58 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
W20260812 07:09:58.158098 13525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:58.158556 13525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:58.158679 13525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:58.165217 13525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:58.165419 13525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:58.165549 13525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:09:58.173566 13525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.156.84:41707/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:09:58.175433 13525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:58.176880 13525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:58.190574 13531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:58.190748 13525 server_base.cc:1034] running on GCE node
W20260812 07:09:58.190577 13532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:58.191200 13534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:58.191859 13525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:58.192745 13525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:58.193953 13525 hybrid_clock.cc:648] HybridClock initialized: now 1786518598193926 us; error 43 us; skew 500 ppm
Aug 12 07:09:58 dist-test-slave-csg5 krb5kdc[12547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518598, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:58.198287 13525 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:09:58.199839 13525 webserver.cc:460] Webserver started at http://127.31.156.67:46345/ using document root <none> and password file <none>
I20260812 07:09:58.200788 13525 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:58.200906 13525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:58.201267 13525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:09:58 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518598, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:09:58.226408 13525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "9ab6a375abc94dad853d1f485986d144"
format_stamp: "Formatted at 2026-08-12 07:09:58 on dist-test-slave-csg5"
server_key: "036ae27a5a91cc9663f7dcf79b36cc56"
server_key_iv: "4de8abc0cfbf4ee434f4a3e21d639f2a"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:58.227437 13525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "9ab6a375abc94dad853d1f485986d144"
format_stamp: "Formatted at 2026-08-12 07:09:58 on dist-test-slave-csg5"
server_key: "036ae27a5a91cc9663f7dcf79b36cc56"
server_key_iv: "4de8abc0cfbf4ee434f4a3e21d639f2a"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:58.256973 13525 fs_manager.cc:691] Time spent creating directory manager: real 0.029s user 0.005s sys 0.000s
I20260812 07:09:58.284588 13544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:58.286690 13525 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.000s
I20260812 07:09:58.286872 13525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "9ab6a375abc94dad853d1f485986d144"
format_stamp: "Formatted at 2026-08-12 07:09:58 on dist-test-slave-csg5"
server_key: "036ae27a5a91cc9663f7dcf79b36cc56"
server_key_iv: "4de8abc0cfbf4ee434f4a3e21d639f2a"
server_key_version: "kuduclusterkey@0"
I20260812 07:09:58.287050 13525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:58.308300 13525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:58.312754 13525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:58.313058 13525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:58.314072 13525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:58.315488 13525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:58.315611 13525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:58.315724 13525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:58.315796 13525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:58.327986 13525 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:46441
I20260812 07:09:58.329407 13525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:09:58.335337 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 13525
I20260812 07:09:58.335484 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:09:58.335812 32369 external_mini_cluster.cc:1428] Setting key 2940c85070bbe6bc49ddf6ddb11ce67c
I20260812 07:09:58.340319 13681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:46441 every 8 connection(s)
Aug 12 07:09:58 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518598, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:09:58.367293 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.341477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:32779 (local address 127.31.156.126:45773)
0812 07:09:58.341628 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:58.341633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:58.343104 (+ 1471us) server_negotiation.cc:407] Connection header received
0812 07:09:58.344498 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:58.344502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:58.344565 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:58.344622 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:58.347758 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.356644 (+ 8886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.358842 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.359172 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.359258 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.362577 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:58.362582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:58.362584 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:58.362614 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:58.364434 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:58.365142 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:58.365146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:58.365148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:58.365387 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:58.365825 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:58.365829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:58.365831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:58.366081 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:58.366161 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:58.366948 (+ 787us) server_negotiation.cc:299] Negotiation successful
0812 07:09:58.367108 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 07:09:58.375912 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.341417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45773 (local address 127.31.156.67:32779)
0812 07:09:58.342918 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:58.342987 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:58.344195 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:58.345097 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:58.345129 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:58.345747 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:58.347522 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.347551 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.356925 (+ 9374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.356929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.358516 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.358534 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.359511 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.359515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.359747 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.360535 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:58.360556 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:58.362369 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:58.364595 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:58.364602 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:58.364625 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:58.365003 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:58.365508 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:58.365511 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:58.365513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:58.365718 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:58.366242 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:58.366251 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:58.366617 (+ 366us) client_negotiation.cc:769] Sending connection context
0812 07:09:58.367778 (+ 1161us) client_negotiation.cc:240] Negotiation successful
0812 07:09:58.368155 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1225,"thread_start_us":193,"threads_started":1}
I20260812 07:09:58.378471 13682 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45773
I20260812 07:09:58.378888 13682 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:58.379560 13682 heartbeater.cc:511] Master 127.31.156.126:45773 requested a full tablet report, sending...
I20260812 07:09:58.380930 13114 ts_manager.cc:194] Registered new tserver with Master: 9ab6a375abc94dad853d1f485986d144 (127.31.156.67:46441)
I20260812 07:09:58.381739 13114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:32779
I20260812 07:09:58.387004 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:09:58.420215 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.399381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45444 (local address 127.31.156.126:45773)
0812 07:09:58.399537 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:58.399543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:58.406261 (+ 6718us) server_negotiation.cc:407] Connection header received
0812 07:09:58.406451 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:58.406454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:58.406520 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:58.406600 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:58.408038 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.408854 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.413676 (+ 4822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.414006 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.414087 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.416543 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:58.416548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:58.416551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:58.416589 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:58.418262 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:58.418810 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:58.418813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:58.418815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:58.418908 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:58.419209 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:58.419212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:58.419231 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:58.419460 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:58.419575 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:58.419923 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 07:09:58.420049 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:09:58.424314 13114 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:09:58.429394 13114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 12 07:09:58 dist-test-slave-csg5 krb5kdc[12547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:09:58.461126 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.442451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:46441 (local address 127.0.0.1:51882)
0812 07:09:58.446167 (+ 3716us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:58.446182 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:58.446323 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:58.448257 (+ 1934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:58.448264 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:58.448290 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:58.448454 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.448462 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.450136 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.450140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.454573 (+ 4433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.454588 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.458312 (+ 3724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.458317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.458471 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.458515 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 07:09:58.460686 (+ 2171us) client_negotiation.cc:240] Negotiation successful
0812 07:09:58.460933 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3649,"thread_start_us":106,"threads_started":1}
I20260812 07:09:58.464942 13702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.442968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.31.156.67:46441)
0812 07:09:58.445297 (+ 2329us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:58.445313 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:58.447531 (+ 2218us) server_negotiation.cc:407] Connection header received
0812 07:09:58.447643 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:58.447649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:58.447910 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:58.448604 (+ 694us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:58.448775 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.449983 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.456853 (+ 6870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.457901 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.458027 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.458229 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:58.464539 (+ 6310us) server_negotiation.cc:299] Negotiation successful
0812 07:09:58.464705 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2188,"thread_start_us":138,"threads_started":1}
I20260812 07:09:58.473058 13600 tablet_service.cc:1467] Processing CreateTablet for tablet 434498448f774fdebcd538fd7c50cc5c (DEFAULT_TABLE table=test-table [id=326999fec12244dcae8fddeb61131ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:58.475574 13600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434498448f774fdebcd538fd7c50cc5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:58.475946 13698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.444558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:42817 (local address 127.0.0.1:47708)
0812 07:09:58.451892 (+ 7334us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:58.451910 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:58.452010 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:58.459737 (+ 7727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:58.459740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:58.459765 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:58.459882 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.459889 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.462940 (+ 3051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.462944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.464149 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.464165 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.475467 (+ 11302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.475473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.475598 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.475631 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:09:58.475684 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 07:09:58.475789 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7239,"thread_start_us":125,"threads_started":1}
I20260812 07:09:58.476696 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.444786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.31.156.66:42817)
0812 07:09:58.458945 (+ 14159us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:58.458953 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:58.458986 (+ 33us) server_negotiation.cc:407] Connection header received
0812 07:09:58.459077 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:58.459083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:58.459369 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:58.459564 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:58.460027 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.462720 (+ 2693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.464340 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.475273 (+ 10933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.476182 (+ 909us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.476376 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:58.476438 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 07:09:58.476549 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14007,"thread_start_us":82,"threads_started":1}
I20260812 07:09:58.480087 13444 tablet_service.cc:1467] Processing CreateTablet for tablet 434498448f774fdebcd538fd7c50cc5c (DEFAULT_TABLE table=test-table [id=326999fec12244dcae8fddeb61131ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:58.481555 13444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434498448f774fdebcd538fd7c50cc5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:58.485046 13703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.440574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:40579 (local address 127.0.0.1:35266)
0812 07:09:58.445608 (+ 5034us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:58.445681 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:58.448125 (+ 2444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:58.453083 (+ 4958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:58.453087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:58.453110 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:58.453238 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.453245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.459312 (+ 6067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.459316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.460542 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.460555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.484466 (+ 23911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.484471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.484601 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.484635 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 07:09:58.484697 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 07:09:58.484793 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4825,"thread_start_us":155,"threads_started":1}
I20260812 07:09:58.486151 13699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.441530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35266 (local address 127.31.156.65:40579)
0812 07:09:58.452315 (+ 10785us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:58.452326 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:58.452356 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:09:58.452449 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:58.452455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:58.452713 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:58.452890 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:58.454757 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.459146 (+ 4389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.463291 (+ 4145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.484265 (+ 20974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.485569 (+ 1304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.485774 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:58.485846 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 07:09:58.485976 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10639,"thread_start_us":89,"threads_started":1}
I20260812 07:09:58.489961 13269 tablet_service.cc:1467] Processing CreateTablet for tablet 434498448f774fdebcd538fd7c50cc5c (DEFAULT_TABLE table=test-table [id=326999fec12244dcae8fddeb61131ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:58.491262 13269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434498448f774fdebcd538fd7c50cc5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:58.497252 13360 heartbeater.cc:503] Master 127.31.156.126:45773 was elected leader, sending a full tablet report...
I20260812 07:09:58.498252 13705 tablet_bootstrap.cc:492] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: Bootstrap starting.
I20260812 07:09:58.500950 13707 tablet_bootstrap.cc:492] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac: Bootstrap starting.
I20260812 07:09:58.503098 13706 tablet_bootstrap.cc:492] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: Bootstrap starting.
I20260812 07:09:58.504523 13705 tablet_bootstrap.cc:654] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:58.505472 13706 tablet_bootstrap.cc:654] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:58.506422 13706 log.cc:824] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:58.507805 13707 tablet_bootstrap.cc:654] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:58.508605 13707 log.cc:824] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:58.507783 13705 log.cc:824] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:58.511109 13706 tablet_bootstrap.cc:492] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: No bootstrap required, opened a new log
I20260812 07:09:58.511358 13706 ts_tablet_manager.cc:1397] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:09:58.515471 13705 tablet_bootstrap.cc:492] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: No bootstrap required, opened a new log
I20260812 07:09:58.515590 13707 tablet_bootstrap.cc:492] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac: No bootstrap required, opened a new log
I20260812 07:09:58.515751 13705 ts_tablet_manager.cc:1397] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.000s
I20260812 07:09:58.515858 13707 ts_tablet_manager.cc:1397] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260812 07:09:58.520437 13705 raft_consensus.cc:348] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 } } peers { permanent_uuid: "9ab6a375abc94dad853d1f485986d144" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 46441 } } peers { permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 } }
I20260812 07:09:58.520707 13705 raft_consensus.cc:374] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:58.520740 13705 raft_consensus.cc:729] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab6a375abc94dad853d1f485986d144, State: Initialized, Role: FOLLOWER
I20260812 07:09:58.521260 13705 consensus_queue.cc:260] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 } } peers { permanent_uuid: "9ab6a375abc94dad853d1f485986d144" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 46441 } } peers { permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 } }
I20260812 07:09:58.521232 13706 raft_consensus.cc:348] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 } } peers { permanent_uuid: "9ab6a375abc94dad853d1f485986d144" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 46441 } } peers { permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 } }
I20260812 07:09:58.521515 13706 raft_consensus.cc:374] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:58.521559 13706 raft_consensus.cc:729] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b923ffafb90549a98505769cdb42ca73, State: Initialized, Role: FOLLOWER
I20260812 07:09:58.521970 13706 consensus_queue.cc:260] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 } } peers { permanent_uuid: "9ab6a375abc94dad853d1f485986d144" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 46441 } } peers { permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 } }
I20260812 07:09:58.522264 13705 ts_tablet_manager.cc:1428] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 07:09:58.522696 13682 heartbeater.cc:503] Master 127.31.156.126:45773 was elected leader, sending a full tablet report...
I20260812 07:09:58.524099 13707 raft_consensus.cc:348] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 } } peers { permanent_uuid: "9ab6a375abc94dad853d1f485986d144" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 46441 } } peers { permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 } }
I20260812 07:09:58.524405 13707 raft_consensus.cc:374] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:58.524454 13707 raft_consensus.cc:729] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d25ffae1fe8406491a9d4ad659c21ac, State: Initialized, Role: FOLLOWER
I20260812 07:09:58.524840 13706 ts_tablet_manager.cc:1428] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260812 07:09:58.524873 13707 consensus_queue.cc:260] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 } } peers { permanent_uuid: "9ab6a375abc94dad853d1f485986d144" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 46441 } } peers { permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 } }
I20260812 07:09:58.525413 13519 heartbeater.cc:503] Master 127.31.156.126:45773 was elected leader, sending a full tablet report...
I20260812 07:09:58.526158 13707 ts_tablet_manager.cc:1428] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
W20260812 07:09:58.589212 13683 tablet.cc:2369] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:09:58.619231 13714 raft_consensus.cc:482] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:58.619410 13716 raft_consensus.cc:482] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:58.619530 13714 raft_consensus.cc:504] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 } } peers { permanent_uuid: "9ab6a375abc94dad853d1f485986d144" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 46441 } } peers { permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 } }
I20260812 07:09:58.619612 13716 raft_consensus.cc:504] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 } } peers { permanent_uuid: "9ab6a375abc94dad853d1f485986d144" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 46441 } } peers { permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 } }
I20260812 07:09:58.620780 13714 leader_election.cc:288] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d25ffae1fe8406491a9d4ad659c21ac (127.31.156.65:40579), b923ffafb90549a98505769cdb42ca73 (127.31.156.66:42817)
I20260812 07:09:58.620903 13716 leader_election.cc:288] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ab6a375abc94dad853d1f485986d144 (127.31.156.67:46441), b923ffafb90549a98505769cdb42ca73 (127.31.156.66:42817)
I20260812 07:09:58.627283 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.621030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:40579 (local address 127.31.156.67:32963)
0812 07:09:58.621682 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:58.621700 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:58.621810 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:58.623505 (+ 1695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:58.623508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:58.623528 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:58.623632 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.623638 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.624614 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.624617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.625813 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.625823 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.626778 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.626782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.626885 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.626945 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 07:09:58.627025 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 07:09:58.627120 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559}
W20260812 07:09:58.627375 13520 tablet.cc:2369] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:09:58.628119 13699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.621857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:32963 (local address 127.31.156.65:40579)
0812 07:09:58.622511 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:58.622517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:58.622531 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:09:58.622563 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:58.622566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:58.622604 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:58.622709 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:58.623763 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.624479 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.625950 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.626539 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.626636 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.626715 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:58.627889 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0812 07:09:58.627979 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":587}
I20260812 07:09:58.628731 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434498448f774fdebcd538fd7c50cc5c" candidate_uuid: "9ab6a375abc94dad853d1f485986d144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" is_pre_election: true
I20260812 07:09:58.628988 13307 raft_consensus.cc:2454] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab6a375abc94dad853d1f485986d144 in term 0.
I20260812 07:09:58.629534 13548 leader_election.cc:302] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d25ffae1fe8406491a9d4ad659c21ac, 9ab6a375abc94dad853d1f485986d144; no voters:
I20260812 07:09:58.629930 13714 raft_consensus.cc:2781] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:09:58.630021 13714 raft_consensus.cc:482] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:09:58.630074 13714 raft_consensus.cc:3037] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:58.631618 13714 raft_consensus.cc:504] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 } } peers { permanent_uuid: "9ab6a375abc94dad853d1f485986d144" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 46441 } } peers { permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 } }
I20260812 07:09:58.632082 13714 leader_election.cc:288] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [CANDIDATE]: Term 1 election: Requested vote from peers 5d25ffae1fe8406491a9d4ad659c21ac (127.31.156.65:40579), b923ffafb90549a98505769cdb42ca73 (127.31.156.66:42817)
I20260812 07:09:58.632499 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.622919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:47877 (local address 127.31.156.66:42817)
0812 07:09:58.623329 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:58.623334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:58.627587 (+ 4253us) server_negotiation.cc:407] Connection header received
0812 07:09:58.627718 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:58.627722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:58.627779 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:58.627852 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:58.628211 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.628884 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.630783 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.631415 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.631528 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.631594 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:58.632280 (+ 686us) server_negotiation.cc:299] Negotiation successful
0812 07:09:58.632374 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337}
I20260812 07:09:58.632576 13307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434498448f774fdebcd538fd7c50cc5c" candidate_uuid: "9ab6a375abc94dad853d1f485986d144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d25ffae1fe8406491a9d4ad659c21ac"
I20260812 07:09:58.632720 13307 raft_consensus.cc:3037] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:58.633327 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.621141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:42817 (local address 127.31.156.67:47877)
0812 07:09:58.627503 (+ 6362us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:58.627519 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:58.627616 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:58.627965 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:58.627968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:58.627985 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:58.628061 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.628067 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.629326 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.629329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.630600 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.630610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.631709 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.631712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.631886 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.631909 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:09:58.631984 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 07:09:58.632065 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6302,"thread_start_us":151,"threads_started":1}
I20260812 07:09:58.634300 13307 raft_consensus.cc:2454] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab6a375abc94dad853d1f485986d144 in term 1.
I20260812 07:09:58.634322 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434498448f774fdebcd538fd7c50cc5c" candidate_uuid: "9ab6a375abc94dad853d1f485986d144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b923ffafb90549a98505769cdb42ca73" is_pre_election: true
I20260812 07:09:58.634673 13466 raft_consensus.cc:2454] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab6a375abc94dad853d1f485986d144 in term 0.
I20260812 07:09:58.635056 13465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434498448f774fdebcd538fd7c50cc5c" candidate_uuid: "9ab6a375abc94dad853d1f485986d144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b923ffafb90549a98505769cdb42ca73"
I20260812 07:09:58.635191 13465 raft_consensus.cc:3037] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:58.635983 13548 leader_election.cc:302] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d25ffae1fe8406491a9d4ad659c21ac, 9ab6a375abc94dad853d1f485986d144; no voters:
I20260812 07:09:58.636188 13714 raft_consensus.cc:2781] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:58.636431 13714 raft_consensus.cc:686] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 1 LEADER]: Becoming Leader. State: Replica: 9ab6a375abc94dad853d1f485986d144, State: Running, Role: LEADER
I20260812 07:09:58.636647 13465 raft_consensus.cc:2454] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab6a375abc94dad853d1f485986d144 in term 1.
I20260812 07:09:58.636865 13714 consensus_queue.cc:237] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 } } peers { permanent_uuid: "9ab6a375abc94dad853d1f485986d144" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 46441 } } peers { permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 } }
I20260812 07:09:58.643623 13114 catalog_manager.cc:5495] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ab6a375abc94dad853d1f485986d144 (127.31.156.67). New cstate: current_term: 1 leader_uuid: "9ab6a375abc94dad853d1f485986d144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ab6a375abc94dad853d1f485986d144" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 46441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 } health_report { overall_health: UNKNOWN } } }
I20260812 07:09:58.649731 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.623036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:36835 (local address 127.31.156.66:42817)
0812 07:09:58.632677 (+ 9641us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:58.632681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:58.632696 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:09:58.632741 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:58.632745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:58.632794 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:58.632874 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:58.639173 (+ 6299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.640000 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.647897 (+ 7897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.648746 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.648886 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.648951 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:58.649430 (+ 479us) server_negotiation.cc:299] Negotiation successful
0812 07:09:58.649539 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9575,"thread_start_us":136,"threads_started":1}
I20260812 07:09:58.650321 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.621312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:42817 (local address 127.31.156.65:36835)
0812 07:09:58.622045 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:58.622060 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:58.622154 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:58.638772 (+ 16618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:58.638777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:58.638812 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:58.638962 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.638970 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.640164 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.640168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.641402 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.641416 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.649103 (+ 7687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.649107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.649240 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.649270 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:09:58.649993 (+ 723us) client_negotiation.cc:240] Negotiation successful
0812 07:09:58.650110 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":120,"threads_started":1}
I20260812 07:09:58.650929 13465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434498448f774fdebcd538fd7c50cc5c" candidate_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b923ffafb90549a98505769cdb42ca73" is_pre_election: true
I20260812 07:09:58.651187 13465 raft_consensus.cc:2379] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5d25ffae1fe8406491a9d4ad659c21ac in current term 1: Already voted for candidate 9ab6a375abc94dad853d1f485986d144 in this term.
I20260812 07:09:58.652592 13702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.621753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:46147 (local address 127.31.156.67:46441)
0812 07:09:58.622404 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:58.622410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:58.622426 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:09:58.622562 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:58.622565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:58.622610 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:58.622684 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:58.623133 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.623882 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.625494 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.626081 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.626173 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.626228 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:58.652331 (+ 26103us) server_negotiation.cc:299] Negotiation successful
0812 07:09:58.652451 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564}
I20260812 07:09:58.653136 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.621386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:46441 (local address 127.31.156.65:46147)
0812 07:09:58.622340 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:58.622354 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:58.622435 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:58.622845 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:58.622847 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:58.622862 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:58.622996 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.623003 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.624035 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.624038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.625352 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:58.625363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.626338 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.626341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:58.652119 (+ 25778us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.652149 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:09:58.652846 (+ 697us) client_negotiation.cc:240] Negotiation successful
0812 07:09:58.652954 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260812 07:09:58.653849 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434498448f774fdebcd538fd7c50cc5c" candidate_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ab6a375abc94dad853d1f485986d144" is_pre_election: true
I20260812 07:09:58.654714 13234 leader_election.cc:302] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d25ffae1fe8406491a9d4ad659c21ac; no voters: 9ab6a375abc94dad853d1f485986d144, b923ffafb90549a98505769cdb42ca73
I20260812 07:09:58.655495 13716 raft_consensus.cc:2726] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 07:09:58.691700 13702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.686859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51898 (local address 127.31.156.67:46441)
0812 07:09:58.686992 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:58.686996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:58.687225 (+ 229us) server_negotiation.cc:407] Connection header received
0812 07:09:58.687421 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:58.687424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:58.687500 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:58.687614 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:58.687907 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.688687 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.689722 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:58.690083 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:58.690196 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:58.690586 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:58.691241 (+ 655us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:58.691376 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:58.691447 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 07:09:58.691510 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:09:58.700052 13465 raft_consensus.cc:1261] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73 [term 1 FOLLOWER]: Refusing update from remote peer 9ab6a375abc94dad853d1f485986d144: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:58.700917 13714 consensus_queue.cc:1035] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b923ffafb90549a98505769cdb42ca73" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 42817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:58.705157 13307 raft_consensus.cc:1261] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac [term 1 FOLLOWER]: Refusing update from remote peer 9ab6a375abc94dad853d1f485986d144: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:58.706153 13714 consensus_queue.cc:1035] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d25ffae1fe8406491a9d4ad659c21ac" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 40579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 07:09:58.708283 13361 tablet.cc:2369] T 434498448f774fdebcd538fd7c50cc5c P 5d25ffae1fe8406491a9d4ad659c21ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:09:58.732161 13727 mvcc.cc:205] Tried to move back new op lower bound from 7317580180265103360 to 7317580180020129792. Current Snapshot: MvccSnapshot[applied={T|T < 7317580180265103360}]
I20260812 07:09:58.744199 13114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:58.744448 13114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:58.748342 13114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e2bc3480baf4517bdcf9fb903655b9b: Sending DeleteTablet for 3 replicas of tablet 434498448f774fdebcd538fd7c50cc5c
I20260812 07:09:58.749636 13729 mvcc.cc:205] Tried to move back new op lower bound from 7317580180265103360 to 7317580180020129792. Current Snapshot: MvccSnapshot[applied={T|T < 7317580180265103360}]
I20260812 07:09:58.750465 13269 tablet_service.cc:1514] Processing DeleteTablet for tablet 434498448f774fdebcd538fd7c50cc5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:58 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:35266
I20260812 07:09:58.750854 13600 tablet_service.cc:1514] Processing DeleteTablet for tablet 434498448f774fdebcd538fd7c50cc5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:58 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:51882
I20260812 07:09:58.751626 13444 tablet_service.cc:1514] Processing DeleteTablet for tablet 434498448f774fdebcd538fd7c50cc5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:58 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:47708
I20260812 07:09:58.753087 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 13207
I20260812 07:09:58.760900 13735 tablet_replica.cc:330] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: stopping tablet replica
I20260812 07:09:58.761413 13735 raft_consensus.cc:2229] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:09:58.762099 13735 raft_consensus.cc:2258] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:58.762547 13736 tablet_replica.cc:330] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: stopping tablet replica
I20260812 07:09:58.762929 13736 raft_consensus.cc:2229] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:58.763311 13736 raft_consensus.cc:2258] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:58.764269 13735 ts_tablet_manager.cc:1905] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:58.767686 13735 ts_tablet_manager.cc:1918] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:09:58.770013 13736 ts_tablet_manager.cc:1905] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:58.774027 13736 ts_tablet_manager.cc:1918] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:09:58.774189 13736 log.cc:1196] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/434498448f774fdebcd538fd7c50cc5c
I20260812 07:09:58.774590 13736 ts_tablet_manager.cc:1939] T 434498448f774fdebcd538fd7c50cc5c P b923ffafb90549a98505769cdb42ca73: Deleting consensus metadata
I20260812 07:09:58.774586 13735 log.cc:1196] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/434498448f774fdebcd538fd7c50cc5c
I20260812 07:09:58.775142 13735 ts_tablet_manager.cc:1939] T 434498448f774fdebcd538fd7c50cc5c P 9ab6a375abc94dad853d1f485986d144: Deleting consensus metadata
I20260812 07:09:58.776866 13093 catalog_manager.cc:4841] TS 9ab6a375abc94dad853d1f485986d144 (127.31.156.67:46441): tablet 434498448f774fdebcd538fd7c50cc5c (table test-table [id=326999fec12244dcae8fddeb61131ef0]) successfully deleted
W20260812 07:09:58.778067 13092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:40579 (error 108)
I20260812 07:09:58.779484 13703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.778954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:40579 (local address 127.0.0.1:35280)
0812 07:09:58.779280 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:58.779387 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:40579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
W20260812 07:09:58.779789 13092 catalog_manager.cc:4584] TS 5d25ffae1fe8406491a9d4ad659c21ac (127.31.156.65:40579): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 434498448f774fdebcd538fd7c50cc5c: Network error: Client connection negotiation failed: client connection to 127.31.156.65:40579: connect: Connection refused (error 111)
I20260812 07:09:58.782933 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 13367
I20260812 07:09:58.800437 13703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:58.800050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:42817 (local address 127.0.0.1:47716)
0812 07:09:58.800263 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:58.800327 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.66:42817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260812 07:09:58.801072 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 13525
I20260812 07:09:58.812798 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 13067
I20260812 07:09:58.824956 32369 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 07:09:59.171247 32369 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 07:09:59.171428 32369 mini_ranger.cc:66] Stopping Ranger...
I20260812 07:09:59.518031 32369 mini_ranger.cc:68] Stopped Ranger
2026-08-12 07:09:59.518 UTC [12581] LOG: received smart shutdown request
2026-08-12 07:09:59.524 UTC [12581] LOG: background worker "logical replication launcher" (PID 12594) exited with exit code 1
2026-08-12 07:09:59.524 UTC [12585] LOG: shutting down
2026-08-12 07:09:59.780 UTC [12581] LOG: database system is shut down
2026-08-12T07:09:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (58636 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:09:59 dist-test-slave-csg5 krb5kdc[13747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:09:59 dist-test-slave-csg5 krb5kdc[13747](info): set up 2 sockets
Aug 12 07:09:59 dist-test-slave-csg5 krb5kdc[13747](info): commencing operation
krb5kdc: starting...
W20260812 07:10:01.236366 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:10:01 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:10:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:01Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.156.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.156.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/rangeradmin_127.31.156.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/rangeradmin_127.31.156.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.156.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.156.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/rangerlookup_127.31.156.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/rangerlookup_127.31.156.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.156.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.156.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/HTTP_127.31.156.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/HTTP_127.31.156.84@KRBTEST.COM.keytab.
I20260812 07:10:01.488366 32369 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskAFNoqi/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/postgres -l logfile start
2026-08-12 07:10:05.831 UTC [13784] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 07:10:05.831 UTC [13784] LOG: listening on IPv4 address "127.31.156.84", port 41331
2026-08-12 07:10:05.837 UTC [13784] LOG: listening on Unix socket "/tmp/.s.PGSQL.41331"
2026-08-12 07:10:05.882 UTC [13787] LOG: database system was shut down at 2026-08-12 07:10:03 UTC
2026-08-12 07:10:05.888 UTC [13784] LOG: database system is ready to accept connections
I20260812 07:10:06.980937 32369 mini_postgres.cc:96] Postgres bound to 41331
2026-08-12 07:10:06.990 UTC [13797] FATAL: database "slave" does not exist
127.31.156.84:41331 - accepting connections
I20260812 07:10:06.990787 32369 mini_ranger.cc:161] Starting Ranger...
I20260812 07:10:07.010941 32369 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 07:10:07.848850 32369 mini_ranger.cc:87] Created ranger Postgres database
I20260812 07:10:07.848956 32369 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-admin
2026-08-12 07:10:08,178 [I] DB FLAVOR :POSTGRES
2026-08-12 07:10:08,178 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:10:08,178 [I] Checking connection..
2026-08-12 07:10:08,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 07:10:08,484 [I] Checking connection passed.
2026-08-12 07:10:08,484 [I] --------- Verifying version history table ---------
2026-08-12 07:10:08,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:10:08,778 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:10:08,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:10:09,076 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:10:09,076 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 07:10:09,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 07:10:09.378 UTC [13873] WARNING: there is no transaction in progress
2026-08-12 07:10:09,387 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 07:10:09,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:10:09,702 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 07:10:09,702 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:10:09,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 07:10:10,005 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 07:10:10,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-csg5.c.gcp-upstream.internal','N') ;"
2026-08-12 07:10:10,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:10:10,918 [I] Table x_portal_user does not exist in database ranger
2026-08-12 07:10:10,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:10:11,203 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 07:10:11,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:10:11,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:10:11,812 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 07:10:11,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 07:10:13.135 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.158 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.182 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.206 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.234 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.258 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.334 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.344 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.365 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.379 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.392 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.412 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.427 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.437 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.450 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:13.462 UTC [14002] WARNING: there is no transaction in progress
2026-08-12 07:10:14,471 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 07:10:14,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 07:10:14,761 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 07:10:14,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:10:15,062 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 07:10:15,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:10:15,374 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 07:10:15,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:10:15,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:10:15,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-csg5.c.gcp-upstream.internal';"
2026-08-12 07:10:16,223 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 07:10:16,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 07:10:16,518 [I] DB_PATCHES have already been applied
I20260812 07:10:16.525529 32369 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-admin:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskAFNoqi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 07:10:16.525616 32369 mini_ranger.cc:193] Using host: 127.31.156.84
I20260812 07:10:16.530879 32369 mini_ranger.cc:227] Ranger admin URL: http://127.31.156.84:59987
Aug 12, 2026 7:10:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:10:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:10:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 7:10:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 7:10:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 7:10:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 7:10:17 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 07:10:17.792991 32369 mini_ranger.cc:160] Time spent starting Ranger: real 10.802s user 0.000s sys 0.015s
Aug 12, 2026 7:10:41 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 7:10:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 7:10:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 7:10:41 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 7:10:42 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 07:10:42.940212 32369 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.156.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.156.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/rangerkms_127.31.156.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.156.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/rangerkms_127.31.156.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.156.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.156.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.156.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/HTTP_127.31.156.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.156.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/HTTP_127.31.156.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 07:10:43 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518643, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 07:10:43.054641 32369 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 07:10:43.069864 32369 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 07:10:44.374563 32369 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 07:10:44.374706 32369 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-kms
I20260812 07:10:44.374732 32369 mini_ranger_kms.cc:227] Using postgres at 127.31.156.84:41331
2026-08-12 07:10:44,590 [I] DB FLAVOR :POSTGRES
2026-08-12 07:10:44,590 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:10:44,590 [I] Checking connection
2026-08-12 07:10:44,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 07:10:44,877 [I] connection success
2026-08-12 07:10:44,878 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 07:10:44,878 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 07:10:44,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 07:10:45,171 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 07:10:45,171 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:10:45,172 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 07:10:45,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.156.84:41331/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 07:10:45,523 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 07:10:46.230201 32369 mini_ranger_kms.cc:310] Created kms service
I20260812 07:10:46.473685 32369 mini_ranger_kms.cc:326] Created kudu user
I20260812 07:10:46.538952 32369 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 07:10:46 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518643, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:10:47.055490 32369 mini_ranger_kms.cc:384] Added ranger policy
I20260812 07:10:47.055675 32369 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-kms:/tmp/dist-test-taskAFNoqi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskAFNoqi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 07:10:47.055701 32369 mini_ranger_kms.cc:240] Using host: 127.31.156.84
I20260812 07:10:47.061784 32369 mini_ranger_kms.cc:276] Ranger KMS PID: 14210
I20260812 07:10:47.061908 32369 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.156.84:44035
Aug 12, 2026 7:10:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:10:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:10:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 7:10:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 7:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 7:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 07:10:49.734543 32369 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.680s user 0.014s sys 0.186s
I20260812 07:10:49.734750 32369 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 07:10:49.734853 32369 mini_ranger_kms.cc:401] 127.31.156.84:44035/kms/v1/keys
Aug 12 07:10:49 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518649, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.156.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 07:10:50 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518649, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.156.84@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
Aug 12 07:10:50 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518649, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.156.84@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
Aug 12 07:10:53 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:10:53.721633 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:40501
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:40501
--ranger_config_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:44035/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:53.850157 14267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:53.850478 14267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:53.850520 14267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:53.855007 14267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:53.855093 14267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:53.855113 14267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:53.855129 14267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:53.855181 14267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:53.861425 14267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.156.84:44035/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:40501
--ranger_config_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:40501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:10:53.862797 14267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:53.863983 14267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:53.873974 14274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:53.874100 14276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:53.873966 14272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:53.874481 14267 server_base.cc:1034] running on GCE node
I20260812 07:10:53.875494 14267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:53.877151 14267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:53.878443 14267 hybrid_clock.cc:648] HybridClock initialized: now 1786518653878421 us; error 46 us; skew 500 ppm
Aug 12 07:10:53 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518653, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:53.882401 14267 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:10:53.883690 14267 webserver.cc:460] Webserver started at http://127.31.156.126:35725/ using document root <none> and password file <none>
I20260812 07:10:53.884320 14267 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:53.884393 14267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:53.884637 14267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:10:53 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518653, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:10:53.940805 14267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "a84d18f0cfd14efa94f338f15379b306"
format_stamp: "Formatted at 2026-08-12 07:10:53 on dist-test-slave-csg5"
server_key: "240c6a1a4c569e49126fba8d7bbf81aa"
server_key_iv: "8eef3a99d09915ea22b1f911e51e763a"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:53.941672 14267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "a84d18f0cfd14efa94f338f15379b306"
format_stamp: "Formatted at 2026-08-12 07:10:53 on dist-test-slave-csg5"
server_key: "240c6a1a4c569e49126fba8d7bbf81aa"
server_key_iv: "8eef3a99d09915ea22b1f911e51e763a"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:53.946365 14267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:10:53.962723 14284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:53.964329 14267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:10:53.964553 14267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "a84d18f0cfd14efa94f338f15379b306"
format_stamp: "Formatted at 2026-08-12 07:10:53 on dist-test-slave-csg5"
server_key: "240c6a1a4c569e49126fba8d7bbf81aa"
server_key_iv: "8eef3a99d09915ea22b1f911e51e763a"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:53.964725 14267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:53.983873 14267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:53.987767 14267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:53.988044 14267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:53.997393 14267 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:40501
I20260812 07:10:53.997432 14345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:40501 every 8 connection(s)
I20260812 07:10:53.998579 14267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:54.001823 14346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:54.002295 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 14267
I20260812 07:10:54.002416 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:10:54.002822 32369 external_mini_cluster.cc:1428] Setting key 0e264030667cb463384590a75195ab80
I20260812 07:10:54.009716 14346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306: Bootstrap starting.
I20260812 07:10:54.012732 14346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:54.013741 14346 log.cc:824] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:10:54 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:10:54.016531 14346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306: No bootstrap required, opened a new log
I20260812 07:10:54.021365 14346 raft_consensus.cc:348] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } }
I20260812 07:10:54.021687 14346 raft_consensus.cc:374] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:54.021783 14346 raft_consensus.cc:729] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84d18f0cfd14efa94f338f15379b306, State: Initialized, Role: FOLLOWER
I20260812 07:10:54.022575 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:54.004837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60288 (local address 127.31.156.126:40501)
0812 07:10:54.005457 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:54.005476 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:54.007156 (+ 1680us) server_negotiation.cc:407] Connection header received
0812 07:10:54.008384 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:54.008413 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:54.008898 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:54.009299 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:54.010235 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:54.011897 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:54.013184 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:54.013929 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:54.014100 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:54.017229 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:54.017235 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:54.017251 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:54.017284 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:54.019210 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:54.019738 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:54.019744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:54.019750 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:54.019878 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:54.020192 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:54.020197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:54.020199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:54.020658 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:54.020914 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:54.021206 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 07:10:54.021551 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":174,"threads_started":1}
I20260812 07:10:54.022647 14346 consensus_queue.cc:260] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } }
I20260812 07:10:54.022902 14346 raft_consensus.cc:388] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:54.022987 14346 raft_consensus.cc:482] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:54.023095 14346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:54.024377 14346 raft_consensus.cc:504] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } }
I20260812 07:10:54.024778 14346 leader_election.cc:302] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a84d18f0cfd14efa94f338f15379b306; no voters:
I20260812 07:10:54.025166 14346 leader_election.cc:288] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:54.025343 14352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:54.025607 14352 raft_consensus.cc:686] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 1 LEADER]: Becoming Leader. State: Replica: a84d18f0cfd14efa94f338f15379b306, State: Running, Role: LEADER
I20260812 07:10:54.025959 14352 consensus_queue.cc:237] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } }
I20260812 07:10:54.026567 14346 sys_catalog.cc:564] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:54.027238 14352 sys_catalog.cc:455] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a84d18f0cfd14efa94f338f15379b306. Latest consensus state: current_term: 1 leader_uuid: "a84d18f0cfd14efa94f338f15379b306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } } }
I20260812 07:10:54.027475 14352 sys_catalog.cc:458] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:54.027765 14353 sys_catalog.cc:455] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a84d18f0cfd14efa94f338f15379b306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } } }
I20260812 07:10:54.027921 14353 sys_catalog.cc:458] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:54.027993 14362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:54.031774 14362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:54.035657 14346 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 07:10:54.038836 14362 catalog_manager.cc:1347] Generated new cluster ID: cacc7ee75efd4ca49d008679488cd217
I20260812 07:10:54.038919 14362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:54.063241 14362 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:54.065512 14362 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:54.078444 14362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306: Generated new TSK 0
I20260812 07:10:54.079799 14362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:10:54.967083 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:44035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40501
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:55.120970 14398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:55.121515 14398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:55.121656 14398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:55.127575 14398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:55.127751 14398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:55.127889 14398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:10:55.137125 14398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.156.84:44035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:10:55.138621 14398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:55.140344 14398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:55.152123 14408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:55.152266 14406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:55.152042 14410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:55.152717 14398 server_base.cc:1034] running on GCE node
I20260812 07:10:55.153220 14398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:55.154230 14398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:55.155422 14398 hybrid_clock.cc:648] HybridClock initialized: now 1786518655155390 us; error 60 us; skew 500 ppm
Aug 12 07:10:55 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518655, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:55.158644 14398 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:10:55.159741 14398 webserver.cc:460] Webserver started at http://127.31.156.65:43735/ using document root <none> and password file <none>
I20260812 07:10:55.160537 14398 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:55.160640 14398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:55.160919 14398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:10:55 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518655, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:10:55.181727 14398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "e763b63992fa43a19a177cef67bba9eb"
format_stamp: "Formatted at 2026-08-12 07:10:55 on dist-test-slave-csg5"
server_key: "fdf7c8ab444e13abe14be0bbc30366a8"
server_key_iv: "87988e8de71b1cfdcc5d7e58a2adfd4b"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:55.182799 14398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "e763b63992fa43a19a177cef67bba9eb"
format_stamp: "Formatted at 2026-08-12 07:10:55 on dist-test-slave-csg5"
server_key: "fdf7c8ab444e13abe14be0bbc30366a8"
server_key_iv: "87988e8de71b1cfdcc5d7e58a2adfd4b"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:55.188592 14398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:10:55.206512 14418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:55.208822 14398 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260812 07:10:55.209033 14398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "e763b63992fa43a19a177cef67bba9eb"
format_stamp: "Formatted at 2026-08-12 07:10:55 on dist-test-slave-csg5"
server_key: "fdf7c8ab444e13abe14be0bbc30366a8"
server_key_iv: "87988e8de71b1cfdcc5d7e58a2adfd4b"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:55.209231 14398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:55.230219 14398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:55.235425 14398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:55.235749 14398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:55.236649 14398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:55.237957 14398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:55.238087 14398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 07:10:55.238229 14398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:55.238333 14398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:55.251426 14398 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:35035
I20260812 07:10:55.253141 14398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:55.254806 14552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:35035 every 8 connection(s)
I20260812 07:10:55.255383 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 14398
I20260812 07:10:55.255507 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:10:55.255831 32369 external_mini_cluster.cc:1428] Setting key d7dde2816e643981cb61ca91e9294c82
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
Aug 12 07:10:55 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518655, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:10:55.301831 14560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:55.267979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40501 (local address 127.31.156.65:37309)
0812 07:10:55.277400 (+ 9421us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:55.277520 (+ 120us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:55.279293 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:55.280374 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:55.280386 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:55.280816 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:55.281734 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:55.281762 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:55.283340 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:55.283344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:55.288251 (+ 4907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:55.288277 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:55.289216 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:55.289222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:55.289545 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:55.290445 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:55.290470 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:55.292694 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:55.296342 (+ 3648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:55.296354 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:55.296384 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:55.296869 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:55.297301 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:55.297305 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:55.297306 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:55.297550 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:55.298238 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:55.298247 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:55.298719 (+ 472us) client_negotiation.cc:769] Sending connection context
0812 07:10:55.298975 (+ 256us) client_negotiation.cc:240] Negotiation successful
0812 07:10:55.299725 (+ 750us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8981,"thread_start_us":247,"threads_started":1}
I20260812 07:10:55.303673 14561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:55.269501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:37309 (local address 127.31.156.126:40501)
0812 07:10:55.279849 (+ 10348us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:55.279853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:55.279867 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:10:55.279908 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:55.279910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:55.279957 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:55.280090 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:55.282201 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:55.283129 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:55.288535 (+ 5406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:55.288945 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:55.290039 (+ 1094us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:55.293170 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:55.293175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:55.293179 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:55.293228 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:55.296090 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:55.297026 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:55.297030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:55.297032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:55.297166 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:55.297685 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:55.297689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:55.297692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:55.297982 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:55.298086 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:55.303122 (+ 5036us) server_negotiation.cc:299] Negotiation successful
0812 07:10:55.303342 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10234,"thread_start_us":101,"threads_started":1}
I20260812 07:10:55.305975 14554 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40501
I20260812 07:10:55.306485 14554 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:55.307703 14554 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
I20260812 07:10:55.310374 14306 ts_manager.cc:194] Registered new tserver with Master: e763b63992fa43a19a177cef67bba9eb (127.31.156.65:35035)
I20260812 07:10:55.312198 14306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:37309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:10:55.328740 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:44035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40501
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:10:55 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518655, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 07:10:55.538447 14563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:55.538990 14563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:55.539068 14563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:55.546582 14563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:55.546744 14563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:55.546851 14563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:10:55.557004 14563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.156.84:44035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:10:55.558917 14563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:55.560501 14563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:55.574581 14575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:55.574615 14573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:55.574618 14572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:55.575232 14563 server_base.cc:1034] running on GCE node
I20260812 07:10:55.575747 14563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:55.576704 14563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:55.584239 14563 hybrid_clock.cc:648] HybridClock initialized: now 1786518655584143 us; error 77 us; skew 500 ppm
Aug 12 07:10:55 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518655, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:55.589174 14563 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:10:55.590631 14563 webserver.cc:460] Webserver started at http://127.31.156.66:44847/ using document root <none> and password file <none>
I20260812 07:10:55.591555 14563 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:55.591677 14563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:55.592074 14563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:10:55 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518655, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:10:55.614199 14563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "5de53e8363d643b999af35a237cfce27"
format_stamp: "Formatted at 2026-08-12 07:10:55 on dist-test-slave-csg5"
server_key: "5f192e226d2707d4bb94eab48e8e5985"
server_key_iv: "1188a7562f03e2786a923486f31a155a"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:55.615324 14563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "5de53e8363d643b999af35a237cfce27"
format_stamp: "Formatted at 2026-08-12 07:10:55 on dist-test-slave-csg5"
server_key: "5f192e226d2707d4bb94eab48e8e5985"
server_key_iv: "1188a7562f03e2786a923486f31a155a"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:55.621872 14563 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 07:10:55.653241 14584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:55.655543 14563 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260812 07:10:55.655838 14563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "5de53e8363d643b999af35a237cfce27"
format_stamp: "Formatted at 2026-08-12 07:10:55 on dist-test-slave-csg5"
server_key: "5f192e226d2707d4bb94eab48e8e5985"
server_key_iv: "1188a7562f03e2786a923486f31a155a"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:55.656147 14563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:55.684558 14563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:55.691524 14563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:55.691808 14563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:55.692621 14563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:55.694510 14563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:55.694607 14563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:55.694816 14563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:55.694900 14563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:55.707576 14563 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:46375
I20260812 07:10:55.707823 14721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:46375 every 8 connection(s)
I20260812 07:10:55.708989 14563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:55.714327 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 14563
I20260812 07:10:55.714745 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:10:55.715085 32369 external_mini_cluster.cc:1428] Setting key 75330408470d2dfe91bec09ea4a473af
Aug 12 07:10:55 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518655, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:10:55.733537 14561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:55.711976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:41695 (local address 127.31.156.126:40501)
0812 07:10:55.712135 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:55.712141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:55.713509 (+ 1368us) server_negotiation.cc:407] Connection header received
0812 07:10:55.715014 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:55.715018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:55.715113 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:55.715177 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:55.717442 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:55.718367 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:55.720611 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:55.721008 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:55.721102 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:55.726585 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:55.726592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:55.726597 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:55.726694 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:55.729477 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:55.730548 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:55.730555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:55.730558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:55.730757 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:55.731443 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:55.731450 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:55.731453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:55.731802 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:55.731946 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:55.733082 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0812 07:10:55.733318 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:10:55.735751 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:55.712555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40501 (local address 127.31.156.66:41695)
0812 07:10:55.713295 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:55.713369 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:55.714690 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:55.715708 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:55.715719 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:55.716159 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:55.717000 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:55.717025 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:55.718606 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:55.718611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:55.720402 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:55.720420 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:55.721394 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:55.721399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:55.721639 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:55.722870 (+ 1231us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:55.722905 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:55.726238 (+ 3333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:55.729737 (+ 3499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:55.729747 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:55.729773 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:55.730349 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:55.730946 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:55.730950 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:55.730954 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:55.731269 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:55.732060 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:55.732072 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:55.732609 (+ 537us) client_negotiation.cc:769] Sending connection context
0812 07:10:55.733790 (+ 1181us) client_negotiation.cc:240] Negotiation successful
0812 07:10:55.734292 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":193,"threads_started":1}
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:10:55.745743 14722 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40501
I20260812 07:10:55.746147 14722 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:55.747023 14722 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
I20260812 07:10:55.748617 14306 ts_manager.cc:194] Registered new tserver with Master: 5de53e8363d643b999af35a237cfce27 (127.31.156.66:46375)
I20260812 07:10:55.749482 14306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:41695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:10:55.795567 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:44035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40501
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:56.033376 14729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:56.033922 14729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:56.034029 14729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:56.041215 14729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:56.041417 14729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:56.041563 14729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:10:56.052318 14729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.156.84:44035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:10:56.054314 14729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:56.055982 14729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 12 07:10:56 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518655, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
W20260812 07:10:56.081441 14737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:56.081533 14735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:56.081717 14734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:56.083117 14729 server_base.cc:1034] running on GCE node
I20260812 07:10:56.083745 14729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:56.084734 14729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:56.088043 14729 hybrid_clock.cc:648] HybridClock initialized: now 1786518656087977 us; error 71 us; skew 500 ppm
Aug 12 07:10:56 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:56.093068 14729 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:10:56.094776 14729 webserver.cc:460] Webserver started at http://127.31.156.67:43991/ using document root <none> and password file <none>
I20260812 07:10:56.095690 14729 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:56.095875 14729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:56.096215 14729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:10:56 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:10:56.122306 14729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "c52debc25d37497d8b51ad8d7cd153c5"
format_stamp: "Formatted at 2026-08-12 07:10:56 on dist-test-slave-csg5"
server_key: "c811e9cac77bb97628e76e9e0f1552a2"
server_key_iv: "bad0dcd65d414c8ed6d70df31660bc93"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:56.129431 14729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "c52debc25d37497d8b51ad8d7cd153c5"
format_stamp: "Formatted at 2026-08-12 07:10:56 on dist-test-slave-csg5"
server_key: "c811e9cac77bb97628e76e9e0f1552a2"
server_key_iv: "bad0dcd65d414c8ed6d70df31660bc93"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:56.146319 14729 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260812 07:10:56.174458 14744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:56.176631 14729 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260812 07:10:56.176819 14729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "c52debc25d37497d8b51ad8d7cd153c5"
format_stamp: "Formatted at 2026-08-12 07:10:56 on dist-test-slave-csg5"
server_key: "c811e9cac77bb97628e76e9e0f1552a2"
server_key_iv: "bad0dcd65d414c8ed6d70df31660bc93"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:56.176985 14729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:56.212884 14729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:56.229107 14729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:56.229389 14729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:56.230198 14729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:56.231575 14729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:56.231639 14729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:56.231700 14729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:56.231724 14729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:56.246068 14729 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:44895
I20260812 07:10:56.247550 14729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:56.250497 14877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:44895 every 8 connection(s)
I20260812 07:10:56.255087 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 14729
I20260812 07:10:56.255236 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:10:56.255576 32369 external_mini_cluster.cc:1428] Setting key e23bc3e0ed51935c02cd44b4253f7888
Aug 12 07:10:56 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:10:56.284488 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:56.256013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:41663 (local address 127.31.156.126:40501)
0812 07:10:56.262378 (+ 6365us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:56.262384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:56.263232 (+ 848us) server_negotiation.cc:407] Connection header received
0812 07:10:56.264898 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:56.264904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:56.265008 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:56.265145 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:56.267075 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:56.267959 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:56.272068 (+ 4109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:56.272439 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:56.272521 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:56.276818 (+ 4297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:56.276824 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:56.276828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:56.276868 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:56.279104 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:56.280043 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:56.280048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:56.280050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:56.280165 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:56.280680 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:56.280684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:56.280686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:56.280994 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:56.283062 (+ 2068us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:56.284069 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0812 07:10:56.284275 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6241,"thread_start_us":137,"threads_started":1}
I20260812 07:10:56.286798 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:56.255557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40501 (local address 127.31.156.67:41663)
0812 07:10:56.262854 (+ 7297us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:56.262965 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:56.264370 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:56.265561 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:56.265571 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:56.266001 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:56.266845 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:56.266871 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:56.268176 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:56.268179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:56.271886 (+ 3707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:56.271905 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:56.272771 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:56.272775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:56.273021 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:56.273847 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:56.273874 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:56.276511 (+ 2637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:56.279359 (+ 2848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:56.279369 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:56.279396 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:56.279890 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:56.280300 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:56.280303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:56.280305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:56.280530 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:56.283169 (+ 2639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:56.283179 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:56.283669 (+ 490us) client_negotiation.cc:769] Sending connection context
0812 07:10:56.284896 (+ 1227us) client_negotiation.cc:240] Negotiation successful
0812 07:10:56.285425 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6929,"thread_start_us":216,"threads_started":1}
I20260812 07:10:56.289284 14878 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40501
I20260812 07:10:56.289620 14878 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:56.290356 14878 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
I20260812 07:10:56.291980 14306 ts_manager.cc:194] Registered new tserver with Master: c52debc25d37497d8b51ad8d7cd153c5 (127.31.156.67:44895)
I20260812 07:10:56.292830 14306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:41663
I20260812 07:10:56.296356 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:10:56.298092 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 14398
I20260812 07:10:56.314220 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 14563
I20260812 07:10:56.340811 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 14729
I20260812 07:10:56.367612 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 14267
I20260812 07:10:56.377882 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:40501
--webserver_interface=127.31.156.126
--webserver_port=35725
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:40501
--ranger_config_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:44035/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.156.84:44035/kms with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:56.517241 14883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:56.517567 14883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:56.517637 14883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:56.522157 14883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:56.522265 14883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:56.522286 14883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:56.522303 14883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:56.522320 14883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:56.529373 14883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.156.84:44035/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:40501
--ranger_config_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:40501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=35725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:10:56.530758 14883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:56.531936 14883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:56.542251 14888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:56.542399 14890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:56.542271 14892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:56.542902 14883 server_base.cc:1034] running on GCE node
I20260812 07:10:56.544306 14883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:56.545982 14883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:56.547477 14883 hybrid_clock.cc:648] HybridClock initialized: now 1786518656547436 us; error 59 us; skew 500 ppm
Aug 12 07:10:56 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:56.551286 14883 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:10:56.552906 14883 webserver.cc:460] Webserver started at http://127.31.156.126:35725/ using document root <none> and password file <none>
I20260812 07:10:56.553624 14883 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:56.553755 14883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:10:56 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:10:56.635749 14883 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 07:10:56.638816 14900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:56.640652 14883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:10:56.640889 14883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "a84d18f0cfd14efa94f338f15379b306"
format_stamp: "Formatted at 2026-08-12 07:10:53 on dist-test-slave-csg5"
server_key: "240c6a1a4c569e49126fba8d7bbf81aa"
server_key_iv: "8eef3a99d09915ea22b1f911e51e763a"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:56.641624 14883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:56.656742 14883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:56.660490 14883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:56.660744 14883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:56.669857 14956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:40501 every 8 connection(s)
I20260812 07:10:56.669869 14883 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:40501
I20260812 07:10:56.671373 14883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:56.676683 14957 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:10:56.678857 14957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306: Bootstrap starting.
I20260812 07:10:56.680685 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 14883
I20260812 07:10:56.681360 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:44035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:35035
--local_ip_for_outbound_sockets=127.31.156.65
--tserver_master_addrs=127.31.156.126:40501
--webserver_port=43735
--webserver_interface=127.31.156.65
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.156.84:44035/kms with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:10:56.698402 14957 log.cc:824] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:56.706553 14957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:10:56.707113 14957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306: Bootstrap complete.
I20260812 07:10:56.712283 14957 raft_consensus.cc:348] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } }
I20260812 07:10:56.712783 14957 raft_consensus.cc:729] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84d18f0cfd14efa94f338f15379b306, State: Initialized, Role: FOLLOWER
I20260812 07:10:56.713610 14957 consensus_queue.cc:260] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } }
I20260812 07:10:56.713840 14957 raft_consensus.cc:388] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:56.713941 14957 raft_consensus.cc:482] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:56.714085 14957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:10:56.716401 14957 raft_consensus.cc:504] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } }
I20260812 07:10:56.716871 14957 leader_election.cc:302] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a84d18f0cfd14efa94f338f15379b306; no voters:
I20260812 07:10:56.717319 14957 leader_election.cc:288] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:10:56.717432 14961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:10:56.717710 14961 raft_consensus.cc:686] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [term 2 LEADER]: Becoming Leader. State: Replica: a84d18f0cfd14efa94f338f15379b306, State: Running, Role: LEADER
I20260812 07:10:56.718122 14961 consensus_queue.cc:237] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } }
I20260812 07:10:56.718847 14957 sys_catalog.cc:564] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:56.719216 14961 sys_catalog.cc:455] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a84d18f0cfd14efa94f338f15379b306. Latest consensus state: current_term: 2 leader_uuid: "a84d18f0cfd14efa94f338f15379b306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } } }
I20260812 07:10:56.719921 14962 sys_catalog.cc:455] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a84d18f0cfd14efa94f338f15379b306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84d18f0cfd14efa94f338f15379b306" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40501 } } }
I20260812 07:10:56.720070 14961 sys_catalog.cc:458] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:56.720381 14962 sys_catalog.cc:458] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:56.721141 14971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:56.721500 14957 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 07:10:56.729753 14971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:56.731012 14971 catalog_manager.cc:1259] Loaded cluster ID: cacc7ee75efd4ca49d008679488cd217
I20260812 07:10:56.731161 14971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:56.734237 14971 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:56.735571 14971 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306: Loaded TSK: 0
I20260812 07:10:56.737057 14971 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:10:56.857936 14959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:56.858421 14959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:56.858553 14959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:56.866348 14959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:56.866528 14959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:56.866699 14959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:10:56.876685 14959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.156.84:44035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:35035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=43735
--tserver_master_addrs=127.31.156.126:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:10:56.878569 14959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:56.880250 14959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:56.894207 15001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:56.894596 15002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:56.894883 15005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:56.898346 14959 server_base.cc:1034] running on GCE node
I20260812 07:10:56.899053 14959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:56.899874 14959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:56.901125 14959 hybrid_clock.cc:648] HybridClock initialized: now 1786518656901083 us; error 55 us; skew 500 ppm
Aug 12 07:10:56 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:56.904968 14959 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:10:56.906385 14959 webserver.cc:460] Webserver started at http://127.31.156.65:43735/ using document root <none> and password file <none>
I20260812 07:10:56.907755 14959 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:56.907919 14959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:10:56 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:10:56.933378 14959 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:10:56.936612 15015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:56.938342 14959 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:10:56.938519 14959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "e763b63992fa43a19a177cef67bba9eb"
format_stamp: "Formatted at 2026-08-12 07:10:55 on dist-test-slave-csg5"
server_key: "fdf7c8ab444e13abe14be0bbc30366a8"
server_key_iv: "87988e8de71b1cfdcc5d7e58a2adfd4b"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:56.939456 14959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:56.981385 14959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:56.985996 14959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:56.986330 14959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:56.987303 14959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:56.988598 14959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:56.988657 14959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:56.988703 14959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:56.988765 14959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:57.001243 14959 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:35035
I20260812 07:10:57.002630 14959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:57.003175 15140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:35035 every 8 connection(s)
I20260812 07:10:57.005077 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 14959
I20260812 07:10:57.005609 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:44035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:46375
--local_ip_for_outbound_sockets=127.31.156.66
--tserver_master_addrs=127.31.156.126:40501
--webserver_port=44847
--webserver_interface=127.31.156.66
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.156.84:44035/kms with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:10:57 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:10:57.044632 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:57.012435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40501 (local address 127.31.156.65:59237)
0812 07:10:57.020643 (+ 8208us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:57.020749 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:57.022074 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:57.024643 (+ 2569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:57.024655 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:57.025321 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:57.026122 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:57.026148 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.029676 (+ 3528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.029681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:57.031687 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:57.031709 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.033291 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.033297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:57.033535 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:57.034336 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:57.034369 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:57.036323 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:57.039775 (+ 3452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:57.039783 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:57.039802 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:57.040288 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:57.040851 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:57.040855 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:57.040857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:57.041107 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:57.042261 (+ 1154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:57.042269 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:57.042623 (+ 354us) client_negotiation.cc:769] Sending connection context
0812 07:10:57.042886 (+ 263us) client_negotiation.cc:240] Negotiation successful
0812 07:10:57.043319 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7895,"thread_start_us":209,"threads_started":1}
I20260812 07:10:57.047533 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:57.014463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:59237 (local address 127.31.156.126:40501)
0812 07:10:57.019021 (+ 4558us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:57.019051 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:57.020981 (+ 1930us) server_negotiation.cc:407] Connection header received
0812 07:10:57.022954 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:57.022972 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:57.023519 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:57.023925 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:57.026436 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.029373 (+ 2937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.031897 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.032748 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.032970 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:57.036669 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:57.036678 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:57.036698 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:57.036744 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:57.039518 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.040490 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.040498 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.040505 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.040706 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.041267 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.041271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.041273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.042012 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:57.042159 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:57.045647 (+ 3488us) server_negotiation.cc:299] Negotiation successful
0812 07:10:57.046254 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4214,"thread_start_us":160,"threads_started":1}
I20260812 07:10:57.051648 15141 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40501
I20260812 07:10:57.051975 15141 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:57.052698 15141 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
W20260812 07:10:57.055727 15141 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:40501 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:10:57.055886 15141 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:57.056120 15141 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
I20260812 07:10:57.057482 15141 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:57.057716 15141 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
W20260812 07:10:57.058800 15141 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:10:57.175370 15143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:57.175809 15143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:57.175931 15143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:57.180814 15143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:57.180980 15143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:57.181098 15143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:10:57.189343 15143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.156.84:44035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:46375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=44847
--tserver_master_addrs=127.31.156.126:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:10:57.191182 15143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:57.192852 15143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:57.209136 15154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:57.209180 15150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:57.209180 15151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:57.209901 15143 server_base.cc:1034] running on GCE node
I20260812 07:10:57.210481 15143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:57.211642 15143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:57.212879 15143 hybrid_clock.cc:648] HybridClock initialized: now 1786518657212851 us; error 49 us; skew 500 ppm
Aug 12 07:10:57 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:57.217814 15143 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:10:57.219372 15143 webserver.cc:460] Webserver started at http://127.31.156.66:44847/ using document root <none> and password file <none>
I20260812 07:10:57.220204 15143 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:57.220423 15143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:10:57 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:10:57.258337 15143 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.004s sys 0.000s
I20260812 07:10:57.262586 15164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:57.264786 15143 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260812 07:10:57.264979 15143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "5de53e8363d643b999af35a237cfce27"
format_stamp: "Formatted at 2026-08-12 07:10:55 on dist-test-slave-csg5"
server_key: "5f192e226d2707d4bb94eab48e8e5985"
server_key_iv: "1188a7562f03e2786a923486f31a155a"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:57.266181 15143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:57.284767 15143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:57.289333 15143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:57.289700 15143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:57.290576 15143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:57.291813 15143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:57.291887 15143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:57.291952 15143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:57.291983 15143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:57.306244 15143 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:46375
I20260812 07:10:57.306245 15295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:46375 every 8 connection(s)
I20260812 07:10:57.308100 15143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:57.317322 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 15143
I20260812 07:10:57.317952 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.156.84:44035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:44895
--local_ip_for_outbound_sockets=127.31.156.67
--tserver_master_addrs=127.31.156.126:40501
--webserver_port=43991
--webserver_interface=127.31.156.67
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.156.84:44035/kms with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:10:57 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:10:57.334401 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:57.312416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:60243 (local address 127.31.156.126:40501)
0812 07:10:57.312588 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:57.312597 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:57.313183 (+ 586us) server_negotiation.cc:407] Connection header received
0812 07:10:57.314316 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:57.314320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:57.314390 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:57.314490 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:57.316414 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.317009 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.319619 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.320096 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.320206 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:57.324287 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:57.324311 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:57.324315 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:57.324360 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:57.327631 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.331883 (+ 4252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.331889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.331891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.332059 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.332633 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.332638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.332640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.332961 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:57.333084 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:57.333977 (+ 893us) server_negotiation.cc:299] Negotiation successful
0812 07:10:57.334196 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:10:57.337046 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:57.312051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40501 (local address 127.31.156.66:60243)
0812 07:10:57.312910 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:57.312981 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:57.314032 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:57.314796 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:57.314809 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:57.315280 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:57.316159 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:57.316185 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.317188 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.317192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:57.319188 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:57.319208 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.320623 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.320630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:57.320910 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:57.321781 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:57.321810 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:57.323973 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:57.331034 (+ 7061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:57.331045 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:57.331072 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:57.331652 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:57.332225 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:57.332229 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:57.332232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:57.332483 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:57.333173 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:57.333181 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:57.333597 (+ 416us) client_negotiation.cc:769] Sending connection context
0812 07:10:57.335198 (+ 1601us) client_negotiation.cc:240] Negotiation successful
0812 07:10:57.335716 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":214,"threads_started":1}
I20260812 07:10:57.339083 15296 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40501
I20260812 07:10:57.339421 15296 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:57.340219 15296 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
W20260812 07:10:57.342484 15296 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:40501 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:10:57.342690 15296 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:57.342965 15296 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
I20260812 07:10:57.344144 15296 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:57.344400 15296 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
W20260812 07:10:57.345528 15296 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:10:57.488905 15299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:57.489271 15299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:57.489643 15299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:57.494971 15299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:57.495095 15299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:57.495204 15299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:10:57.501981 15299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.156.84:44035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:44895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=43991
--tserver_master_addrs=127.31.156.126:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:10:57.503345 15299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:57.504503 15299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:57.516922 15306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:57.516785 15305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:57.517200 15299 server_base.cc:1034] running on GCE node
W20260812 07:10:57.517365 15308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:57.517947 15299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:57.518956 15299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:57.520161 15299 hybrid_clock.cc:648] HybridClock initialized: now 1786518657520137 us; error 43 us; skew 500 ppm
Aug 12 07:10:57 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:57.524107 15299 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:10:57.525465 15299 webserver.cc:460] Webserver started at http://127.31.156.67:43991/ using document root <none> and password file <none>
I20260812 07:10:57.526160 15299 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:57.526294 15299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:10:57 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:10:57.560523 15299 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.000s
I20260812 07:10:57.564790 15317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:57.566879 15299 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:10:57.567160 15299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "c52debc25d37497d8b51ad8d7cd153c5"
format_stamp: "Formatted at 2026-08-12 07:10:56 on dist-test-slave-csg5"
server_key: "c811e9cac77bb97628e76e9e0f1552a2"
server_key_iv: "bad0dcd65d414c8ed6d70df31660bc93"
server_key_version: "kuduclusterkey@0"
I20260812 07:10:57.568336 15299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:57.590893 15299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:57.594417 15299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:57.594731 15299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:57.595498 15299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:57.596526 15299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:57.596601 15299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:57.596683 15299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:57.596730 15299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:57.610610 15299 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:44895
I20260812 07:10:57.610780 15442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:44895 every 8 connection(s)
I20260812 07:10:57.612109 15299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:57.612748 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 15299
I20260812 07:10:57.626180 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:57.613594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58736 (local address 127.31.156.126:40501)
0812 07:10:57.613764 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:57.613769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:57.615042 (+ 1273us) server_negotiation.cc:407] Connection header received
0812 07:10:57.615547 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:57.615551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:57.615625 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:57.615724 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:57.617533 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.618311 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.619832 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.620194 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.620315 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:57.622384 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:57.622390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:57.622392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:57.622426 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:57.624149 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.624760 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.624765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.624768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.624876 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.625211 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.625214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.625215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.625397 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:57.625551 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:57.625850 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 07:10:57.625983 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 07:10:57 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:10:57.636243 15455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:57.617226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:55607 (local address 127.31.156.126:40501)
0812 07:10:57.618889 (+ 1663us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:57.618898 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:57.618916 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:10:57.619027 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:57.619030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:57.619083 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:57.619194 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:57.620705 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.621281 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.622959 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.623352 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.623463 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:57.627930 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:57.627934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:57.627937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:57.627966 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:57.632619 (+ 4653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.633661 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.633665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.633667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.633811 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.634429 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.634433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.634435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.634872 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:57.634979 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:57.635910 (+ 931us) server_negotiation.cc:299] Negotiation successful
0812 07:10:57.636068 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1585,"thread_start_us":85,"threads_started":1}
I20260812 07:10:57.637264 15454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:57.616140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40501 (local address 127.31.156.67:55607)
0812 07:10:57.617611 (+ 1471us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:57.617684 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:57.618774 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:57.619391 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:57.619399 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:57.619707 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:57.620546 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:57.620564 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.621394 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.621398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:57.622802 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:57.622813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.623546 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.623551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:57.624442 (+ 891us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:57.625672 (+ 1230us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:57.625691 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:57.627689 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:57.632891 (+ 5202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:57.632901 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:57.632927 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:57.633492 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:57.633964 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:57.633968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:57.633970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:57.634226 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:57.635011 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:57.635021 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:57.635429 (+ 408us) client_negotiation.cc:769] Sending connection context
0812 07:10:57.635678 (+ 249us) client_negotiation.cc:240] Negotiation successful
0812 07:10:57.636087 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1208,"thread_start_us":165,"threads_started":1}
I20260812 07:10:57.638924 15443 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40501
I20260812 07:10:57.639314 15443 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:57.639935 15443 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
I20260812 07:10:57.641745 14919 ts_manager.cc:194] Registered new tserver with Master: c52debc25d37497d8b51ad8d7cd153c5 (127.31.156.67:44895)
I20260812 07:10:57.643453 14919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:55607
Aug 12 07:10:57 dist-test-slave-csg5 krb5kdc[13747](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:58.059123 15141 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:58.059617 15141 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
I20260812 07:10:58.060526 14918 ts_manager.cc:194] Registered new tserver with Master: e763b63992fa43a19a177cef67bba9eb (127.31.156.65:35035)
I20260812 07:10:58.061452 14918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:59237
I20260812 07:10:58.345799 15296 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:58.346179 15296 heartbeater.cc:511] Master 127.31.156.126:40501 requested a full tablet report, sending...
I20260812 07:10:58.347249 14919 ts_manager.cc:194] Registered new tserver with Master: 5de53e8363d643b999af35a237cfce27 (127.31.156.66:46375)
I20260812 07:10:58.348090 14919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:60243
I20260812 07:10:58.355582 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:10:58.370471 15467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.357978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58740 (local address 127.31.156.126:40501)
0812 07:10:58.358526 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.358531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.358552 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:10:58.358604 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.358607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.358693 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.358800 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:58.359860 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.360724 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.362035 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.362416 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.362549 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.364696 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:58.364701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:58.364704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:58.364742 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:58.367953 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:58.368657 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:58.368662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:58.368664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:58.368816 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:58.369234 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:58.369238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:58.369239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:58.369542 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:58.369739 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.370109 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.370267 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":120,"threads_started":1}
I20260812 07:10:58.376310 14919 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:10:58.382310 14919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:10:58.409916 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.400680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:44895 (local address 127.0.0.1:32838)
0812 07:10:58.403459 (+ 2779us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.403476 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.403586 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.404665 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.404669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.404690 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:58.404820 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.404827 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.406433 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.406438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.407722 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.407734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.409269 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.409274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.409413 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.409460 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.409576 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.409712 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2695,"thread_start_us":119,"threads_started":1}
I20260812 07:10:58.410778 15480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.400855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32838 (local address 127.31.156.67:44895)
0812 07:10:58.402899 (+ 2044us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.402907 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.403969 (+ 1062us) server_negotiation.cc:407] Connection header received
0812 07:10:58.404072 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.404078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.404313 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.404482 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:58.404979 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.406245 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.407900 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.408895 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.409013 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.409199 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.410445 (+ 1246us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.410574 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1921,"thread_start_us":95,"threads_started":1}
I20260812 07:10:58.415035 15356 tablet_service.cc:1467] Processing CreateTablet for tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8 (DEFAULT_TABLE table=test-table [id=43ba0d1d97b844bbbadcc0177408fc63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:58.417028 15356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:58.417233 15481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.401901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:35035 (local address 127.0.0.1:44518)
0812 07:10:58.408759 (+ 6858us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.408775 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.408903 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.409784 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.409787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.409807 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:58.409913 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.409920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.412638 (+ 2718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.412642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.413911 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.413927 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.416694 (+ 2767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.416698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.416824 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.416857 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.416946 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.417049 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6782,"thread_start_us":123,"threads_started":1}
I20260812 07:10:58.417847 15482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.402511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44518 (local address 127.31.156.65:35035)
0812 07:10:58.408576 (+ 6065us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.408585 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.409034 (+ 449us) server_negotiation.cc:407] Connection header received
0812 07:10:58.409140 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.409147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.409431 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.409616 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:58.410070 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.412461 (+ 2391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.414096 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.416266 (+ 2170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.416416 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.416606 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.417551 (+ 945us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.417691 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5921,"thread_start_us":80,"threads_started":1}
I20260812 07:10:58.422096 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.399146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:46375 (local address 127.0.0.1:55284)
0812 07:10:58.402091 (+ 2945us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.402164 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.402342 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.403176 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.403183 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.403216 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:58.403387 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.403395 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.404889 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.404893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.418185 (+ 13292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.418200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.421565 (+ 3365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.421570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.421683 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.421711 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.421801 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.421901 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2704,"thread_start_us":162,"threads_started":1}
I20260812 07:10:58.421949 15070 tablet_service.cc:1467] Processing CreateTablet for tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8 (DEFAULT_TABLE table=test-table [id=43ba0d1d97b844bbbadcc0177408fc63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:58.422765 15475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.399301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.31.156.66:46375)
0812 07:10:58.399693 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.399701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.402534 (+ 2833us) server_negotiation.cc:407] Connection header received
0812 07:10:58.402648 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.402685 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.402962 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.403152 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:58.403553 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.404740 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.419143 (+ 14403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.421033 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.421313 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.421492 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.422431 (+ 939us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.422570 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":70,"threads_started":1}
I20260812 07:10:58.423560 15070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:58.426354 15204 tablet_service.cc:1467] Processing CreateTablet for tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8 (DEFAULT_TABLE table=test-table [id=43ba0d1d97b844bbbadcc0177408fc63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:58.427628 15204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:58.433761 15484 tablet_bootstrap.cc:492] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: Bootstrap starting.
I20260812 07:10:58.436192 15485 tablet_bootstrap.cc:492] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb: Bootstrap starting.
I20260812 07:10:58.436724 15484 tablet_bootstrap.cc:654] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:58.437821 15484 log.cc:824] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:58.439114 15485 tablet_bootstrap.cc:654] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:58.440200 15485 log.cc:824] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:58.440747 15484 tablet_bootstrap.cc:492] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: No bootstrap required, opened a new log
I20260812 07:10:58.441008 15484 ts_tablet_manager.cc:1397] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
Aug 12 07:10:58 dist-test-slave-csg5 krb5kdc[13747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for HTTP/127.31.156.84@KRBTEST.COM
I20260812 07:10:58.443069 15485 tablet_bootstrap.cc:492] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb: No bootstrap required, opened a new log
I20260812 07:10:58.443306 15485 ts_tablet_manager.cc:1397] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:10:58.446190 15484 raft_consensus.cc:348] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de53e8363d643b999af35a237cfce27" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46375 } } peers { permanent_uuid: "c52debc25d37497d8b51ad8d7cd153c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 44895 } } peers { permanent_uuid: "e763b63992fa43a19a177cef67bba9eb" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35035 } }
I20260812 07:10:58.446584 15484 raft_consensus.cc:374] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:58.446700 15484 raft_consensus.cc:729] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52debc25d37497d8b51ad8d7cd153c5, State: Initialized, Role: FOLLOWER
I20260812 07:10:58.447325 15484 consensus_queue.cc:260] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de53e8363d643b999af35a237cfce27" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46375 } } peers { permanent_uuid: "c52debc25d37497d8b51ad8d7cd153c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 44895 } } peers { permanent_uuid: "e763b63992fa43a19a177cef67bba9eb" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35035 } }
I20260812 07:10:58.448731 15484 ts_tablet_manager.cc:1428] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260812 07:10:58.449257 15443 heartbeater.cc:503] Master 127.31.156.126:40501 was elected leader, sending a full tablet report...
I20260812 07:10:58.449384 15485 raft_consensus.cc:348] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de53e8363d643b999af35a237cfce27" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46375 } } peers { permanent_uuid: "c52debc25d37497d8b51ad8d7cd153c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 44895 } } peers { permanent_uuid: "e763b63992fa43a19a177cef67bba9eb" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35035 } }
I20260812 07:10:58.449620 15490 tablet_bootstrap.cc:492] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27: Bootstrap starting.
I20260812 07:10:58.449708 15485 raft_consensus.cc:374] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:58.449764 15485 raft_consensus.cc:729] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e763b63992fa43a19a177cef67bba9eb, State: Initialized, Role: FOLLOWER
I20260812 07:10:58.452584 15490 tablet_bootstrap.cc:654] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:58.453658 15490 log.cc:824] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:58.450332 15485 consensus_queue.cc:260] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de53e8363d643b999af35a237cfce27" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46375 } } peers { permanent_uuid: "c52debc25d37497d8b51ad8d7cd153c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 44895 } } peers { permanent_uuid: "e763b63992fa43a19a177cef67bba9eb" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35035 } }
I20260812 07:10:58.457535 15485 ts_tablet_manager.cc:1428] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb: Time spent starting tablet: real 0.014s user 0.007s sys 0.001s
I20260812 07:10:58.458030 15490 tablet_bootstrap.cc:492] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27: No bootstrap required, opened a new log
I20260812 07:10:58.458063 15141 heartbeater.cc:503] Master 127.31.156.126:40501 was elected leader, sending a full tablet report...
I20260812 07:10:58.458686 15490 ts_tablet_manager.cc:1397] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 07:10:58.465021 15490 raft_consensus.cc:348] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de53e8363d643b999af35a237cfce27" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46375 } } peers { permanent_uuid: "c52debc25d37497d8b51ad8d7cd153c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 44895 } } peers { permanent_uuid: "e763b63992fa43a19a177cef67bba9eb" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35035 } }
I20260812 07:10:58.465457 15490 raft_consensus.cc:374] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:58.465531 15490 raft_consensus.cc:729] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de53e8363d643b999af35a237cfce27, State: Initialized, Role: FOLLOWER
I20260812 07:10:58.466110 15490 consensus_queue.cc:260] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de53e8363d643b999af35a237cfce27" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46375 } } peers { permanent_uuid: "c52debc25d37497d8b51ad8d7cd153c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 44895 } } peers { permanent_uuid: "e763b63992fa43a19a177cef67bba9eb" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35035 } }
I20260812 07:10:58.468273 15296 heartbeater.cc:503] Master 127.31.156.126:40501 was elected leader, sending a full tablet report...
I20260812 07:10:58.468430 15490 ts_tablet_manager.cc:1428] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
W20260812 07:10:58.521723 15142 tablet.cc:2369] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:10:58.560310 15297 tablet.cc:2369] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:10:58.614238 15444 tablet.cc:2369] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:10:58.620802 15492 raft_consensus.cc:482] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:10:58.621062 15492 raft_consensus.cc:504] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de53e8363d643b999af35a237cfce27" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46375 } } peers { permanent_uuid: "c52debc25d37497d8b51ad8d7cd153c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 44895 } } peers { permanent_uuid: "e763b63992fa43a19a177cef67bba9eb" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35035 } }
I20260812 07:10:58.622565 15492 leader_election.cc:288] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5de53e8363d643b999af35a237cfce27 (127.31.156.66:46375), c52debc25d37497d8b51ad8d7cd153c5 (127.31.156.67:44895)
I20260812 07:10:58.632853 15475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.624048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:47749 (local address 127.31.156.66:46375)
0812 07:10:58.624181 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.624186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.624508 (+ 322us) server_negotiation.cc:407] Connection header received
0812 07:10:58.624756 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.624760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.624810 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.624874 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:58.625353 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.626115 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.631029 (+ 4914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.631888 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.632039 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.632106 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.632550 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.632660 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 07:10:58.633490 15495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.623071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:46375 (local address 127.31.156.65:47749)
0812 07:10:58.624413 (+ 1342us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.624429 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.624627 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.625023 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.625026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.625044 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:58.625212 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.625218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.626282 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.626286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.627664 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.627676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.632232 (+ 4556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.632235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.632374 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.632401 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.633158 (+ 757us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.633269 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1274,"thread_start_us":150,"threads_started":1}
I20260812 07:10:58.634341 15238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcd9dbc26b6d45ee8b1b2bdd79149dc8" candidate_uuid: "e763b63992fa43a19a177cef67bba9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de53e8363d643b999af35a237cfce27" is_pre_election: true
I20260812 07:10:58.634853 15238 raft_consensus.cc:2454] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e763b63992fa43a19a177cef67bba9eb in term 0.
I20260812 07:10:58.635522 15020 leader_election.cc:302] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de53e8363d643b999af35a237cfce27, e763b63992fa43a19a177cef67bba9eb; no voters:
I20260812 07:10:58.635921 15492 raft_consensus.cc:2781] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:10:58.636019 15492 raft_consensus.cc:482] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:10:58.636070 15492 raft_consensus.cc:3037] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:58.637562 15492 raft_consensus.cc:504] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de53e8363d643b999af35a237cfce27" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46375 } } peers { permanent_uuid: "c52debc25d37497d8b51ad8d7cd153c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 44895 } } peers { permanent_uuid: "e763b63992fa43a19a177cef67bba9eb" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35035 } }
I20260812 07:10:58.638036 15492 leader_election.cc:288] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [CANDIDATE]: Term 1 election: Requested vote from peers 5de53e8363d643b999af35a237cfce27 (127.31.156.66:46375), c52debc25d37497d8b51ad8d7cd153c5 (127.31.156.67:44895)
I20260812 07:10:58.638738 15238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcd9dbc26b6d45ee8b1b2bdd79149dc8" candidate_uuid: "e763b63992fa43a19a177cef67bba9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de53e8363d643b999af35a237cfce27"
I20260812 07:10:58.638906 15238 raft_consensus.cc:3037] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:58.640405 15238 raft_consensus.cc:2454] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e763b63992fa43a19a177cef67bba9eb in term 1.
I20260812 07:10:58.640823 15020 leader_election.cc:302] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de53e8363d643b999af35a237cfce27, e763b63992fa43a19a177cef67bba9eb; no voters:
I20260812 07:10:58.641053 15492 raft_consensus.cc:2781] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:58.641321 15492 raft_consensus.cc:686] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [term 1 LEADER]: Becoming Leader. State: Replica: e763b63992fa43a19a177cef67bba9eb, State: Running, Role: LEADER
I20260812 07:10:58.641762 15492 consensus_queue.cc:237] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de53e8363d643b999af35a237cfce27" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46375 } } peers { permanent_uuid: "c52debc25d37497d8b51ad8d7cd153c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 44895 } } peers { permanent_uuid: "e763b63992fa43a19a177cef67bba9eb" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35035 } }
I20260812 07:10:58.645542 15480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.624199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:43261 (local address 127.31.156.67:44895)
0812 07:10:58.624305 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.624310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.628021 (+ 3711us) server_negotiation.cc:407] Connection header received
0812 07:10:58.628282 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.628286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.628338 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.628385 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:58.628761 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.629512 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.634041 (+ 4529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.634807 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.634928 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.634987 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.645238 (+ 10251us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.645383 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 07:10:58.646129 15494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.623556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:44895 (local address 127.31.156.65:43261)
0812 07:10:58.627931 (+ 4375us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.627946 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.628096 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.628518 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.628521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.628537 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:58.628646 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.628651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.629696 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.629699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.630851 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.630863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.635107 (+ 4244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.635110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.644859 (+ 9749us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.644898 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.645817 (+ 919us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.645927 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4294,"thread_start_us":105,"threads_started":1}
I20260812 07:10:58.645824 14919 catalog_manager.cc:5495] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb reported cstate change: term changed from 0 to 1, leader changed from <none> to e763b63992fa43a19a177cef67bba9eb (127.31.156.65). New cstate: current_term: 1 leader_uuid: "e763b63992fa43a19a177cef67bba9eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de53e8363d643b999af35a237cfce27" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c52debc25d37497d8b51ad8d7cd153c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e763b63992fa43a19a177cef67bba9eb" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35035 } health_report { overall_health: HEALTHY } } }
I20260812 07:10:58.646889 15392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcd9dbc26b6d45ee8b1b2bdd79149dc8" candidate_uuid: "e763b63992fa43a19a177cef67bba9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52debc25d37497d8b51ad8d7cd153c5" is_pre_election: true
I20260812 07:10:58.647253 15392 raft_consensus.cc:2454] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e763b63992fa43a19a177cef67bba9eb in term 0.
I20260812 07:10:58.647663 15393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcd9dbc26b6d45ee8b1b2bdd79149dc8" candidate_uuid: "e763b63992fa43a19a177cef67bba9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52debc25d37497d8b51ad8d7cd153c5"
I20260812 07:10:58.647818 15393 raft_consensus.cc:3037] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:58.649426 15393 raft_consensus.cc:2454] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e763b63992fa43a19a177cef67bba9eb in term 1.
I20260812 07:10:58.711194 15482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.704479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44530 (local address 127.31.156.65:35035)
0812 07:10:58.704627 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.704635 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.704666 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:10:58.704755 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.704757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.704950 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.705134 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:10:58.705496 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.706414 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.708324 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.708753 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.708876 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.709476 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:10:58.710172 (+ 696us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:10:58.710315 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.710907 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.710987 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:10:58.721287 15393 raft_consensus.cc:1261] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5 [term 1 FOLLOWER]: Refusing update from remote peer e763b63992fa43a19a177cef67bba9eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:58.721344 15238 raft_consensus.cc:1261] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27 [term 1 FOLLOWER]: Refusing update from remote peer e763b63992fa43a19a177cef67bba9eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:58.722297 15496 consensus_queue.cc:1035] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5de53e8363d643b999af35a237cfce27" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:58.722509 15492 consensus_queue.cc:1035] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P e763b63992fa43a19a177cef67bba9eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c52debc25d37497d8b51ad8d7cd153c5" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:58.743894 15499 mvcc.cc:205] Tried to move back new op lower bound from 7317580426107596800 to 7317580425800515584. Current Snapshot: MvccSnapshot[applied={T|T < 7317580426107596800}]
I20260812 07:10:58.744103 15501 mvcc.cc:205] Tried to move back new op lower bound from 7317580426107596800 to 7317580425800515584. Current Snapshot: MvccSnapshot[applied={T|T < 7317580426107596800}]
I20260812 07:10:58.763026 14919 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:58.763221 14919 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:58.766371 14919 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a84d18f0cfd14efa94f338f15379b306: Sending DeleteTablet for 3 replicas of tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8
I20260812 07:10:58.767428 15204 tablet_service.cc:1514] Processing DeleteTablet for tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:58 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:55284
I20260812 07:10:58.769140 15356 tablet_service.cc:1514] Processing DeleteTablet for tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:58 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:32838
I20260812 07:10:58.770015 15070 tablet_service.cc:1514] Processing DeleteTablet for tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:58 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:44518
I20260812 07:10:58.771219 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 14959
I20260812 07:10:58.779484 15500 mvcc.cc:205] Tried to move back new op lower bound from 7317580426107596800 to 7317580425800515584. Current Snapshot: MvccSnapshot[applied={T|T < 7317580426107596800}]
I20260812 07:10:58.779815 15507 tablet_replica.cc:330] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27: stopping tablet replica
I20260812 07:10:58.780197 15507 raft_consensus.cc:2229] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:58.780542 15507 raft_consensus.cc:2258] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:58.780997 15508 tablet_replica.cc:330] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: stopping tablet replica
I20260812 07:10:58.781329 15508 raft_consensus.cc:2229] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:58.781644 15508 raft_consensus.cc:2258] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:58.784584 15507 ts_tablet_manager.cc:1905] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P 5de53e8363d643b999af35a237cfce27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:58.784595 15508 ts_tablet_manager.cc:1905] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:58.788470 15508 ts_tablet_manager.cc:1918] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:58.788604 15508 log.cc:1196] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/bcd9dbc26b6d45ee8b1b2bdd79149dc8
I20260812 07:10:58.788951 15508 ts_tablet_manager.cc:1939] T bcd9dbc26b6d45ee8b1b2bdd79149dc8 P c52debc25d37497d8b51ad8d7cd153c5: Deleting consensus metadata
W20260812 07:10:58.790191 14902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:35035 (error 108)
I20260812 07:10:58.790835 14902 catalog_manager.cc:4841] TS c52debc25d37497d8b51ad8d7cd153c5 (127.31.156.67:44895): tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8 (table test-table [id=43ba0d1d97b844bbbadcc0177408fc63]) successfully deleted
I20260812 07:10:58.791461 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 15143
I20260812 07:10:58.805235 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.804749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:35035 (local address 127.0.0.1:44532)
0812 07:10:58.804999 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.805095 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:35035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260812 07:10:58.805662 14902 catalog_manager.cc:4584] TS e763b63992fa43a19a177cef67bba9eb (127.31.156.65:35035): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bcd9dbc26b6d45ee8b1b2bdd79149dc8: Network error: Client connection negotiation failed: client connection to 127.31.156.65:35035: connect: Connection refused (error 111)
W20260812 07:10:58.815498 15479 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:58.814813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:46375 (local address 127.0.0.1:55296)
0812 07:10:58.815008 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.815024 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.815123 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.815365 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.66:46375: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":116}
I20260812 07:10:58.816160 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 15299
I20260812 07:10:58.826902 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 14883
I20260812 07:10:58.840454 32369 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 07:10:59.187036 32369 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 07:10:59.187204 32369 mini_ranger.cc:66] Stopping Ranger...
I20260812 07:10:59.543812 32369 mini_ranger.cc:68] Stopped Ranger
2026-08-12 07:10:59.544 UTC [13784] LOG: received smart shutdown request
2026-08-12 07:10:59.550 UTC [13784] LOG: background worker "logical replication launcher" (PID 13795) exited with exit code 1
2026-08-12 07:10:59.550 UTC [13788] LOG: shutting down
2026-08-12 07:10:59.774 UTC [13784] LOG: database system is shut down
2026-08-12T07:10:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (60009 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:10:59 dist-test-slave-csg5 krb5kdc[15522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:10:59 dist-test-slave-csg5 krb5kdc[15522](info): set up 2 sockets
Aug 12 07:10:59 dist-test-slave-csg5 krb5kdc[15522](info): commencing operation
krb5kdc: starting...
W20260812 07:11:01.174907 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:11:01 dist-test-slave-csg5 krb5kdc[15522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:11:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:11:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:11:01.343401 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45693
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:45693,127.31.156.125:35249,127.31.156.124:33913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:01.478727 15537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:01.479096 15537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:01.479175 15537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:01.483619 15537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:11:01.483713 15537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:01.483734 15537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:01.483753 15537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:11:01.483769 15537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:11:01.489959 15537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:45693,127.31.156.125:35249,127.31.156.124:33913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:01.491334 15537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:01.492642 15537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:01.502540 15547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:01.502601 15537 server_base.cc:1034] running on GCE node
W20260812 07:11:01.502445 15544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:01.502494 15545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:01.503796 15537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:01.505141 15537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:01.506428 15537 hybrid_clock.cc:648] HybridClock initialized: now 1786518661506416 us; error 36 us; skew 500 ppm
Aug 12 07:11:01 dist-test-slave-csg5 krb5kdc[15522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:01.510038 15537 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:11:01.511330 15537 webserver.cc:460] Webserver started at http://127.31.156.126:35931/ using document root <none> and password file <none>
I20260812 07:11:01.512006 15537 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:01.512117 15537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:01.512388 15537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:01.514868 15537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "743b733f34be41aeb7ea6a57a47e87a7"
format_stamp: "Formatted at 2026-08-12 07:11:01 on dist-test-slave-csg5"
server_key: "7fb94ca2098c68da76845f8f1ed293aa"
server_key_iv: "aee606ca4e5d9d212afdb2bb7d6c826c"
server_key_version: "encryptionkey@0"
I20260812 07:11:01.515570 15537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "743b733f34be41aeb7ea6a57a47e87a7"
format_stamp: "Formatted at 2026-08-12 07:11:01 on dist-test-slave-csg5"
server_key: "7fb94ca2098c68da76845f8f1ed293aa"
server_key_iv: "aee606ca4e5d9d212afdb2bb7d6c826c"
server_key_version: "encryptionkey@0"
I20260812 07:11:01.520112 15537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:11:01.523352 15555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:01.524830 15537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:01.525019 15537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "743b733f34be41aeb7ea6a57a47e87a7"
format_stamp: "Formatted at 2026-08-12 07:11:01 on dist-test-slave-csg5"
server_key: "7fb94ca2098c68da76845f8f1ed293aa"
server_key_iv: "aee606ca4e5d9d212afdb2bb7d6c826c"
server_key_version: "encryptionkey@0"
I20260812 07:11:01.525183 15537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:01.545926 15537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:01.550024 15537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:01.550316 15537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:01.558768 15537 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:45693
I20260812 07:11:01.558913 15613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:45693 every 8 connection(s)
I20260812 07:11:01.559907 15537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:11:01.563159 15614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:01.563939 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 15537
I20260812 07:11:01.564092 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:11:01.564440 32369 external_mini_cluster.cc:1428] Setting key 5593668823a642f05cae75a534f8b980
I20260812 07:11:01.568693 15614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.156.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.125@KRBTEST.COM" created.
Aug 12 07:11:01 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:01.588444 15618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.572972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54750 (local address 127.31.156.126:45693)
0812 07:11:01.573395 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:01.573407 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:01.573457 (+ 50us) server_negotiation.cc:407] Connection header received
0812 07:11:01.573961 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:01.573966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:01.574177 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:01.574389 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:01.575364 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.576376 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.577452 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.577981 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.578175 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:01.581817 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:01.581828 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:01.581857 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:01.581897 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:01.584309 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:01.585162 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:01.585175 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:01.585176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:01.585408 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:01.585889 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:01.585892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:01.585893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:01.586457 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:01.586587 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:01.587185 (+ 598us) server_negotiation.cc:299] Negotiation successful
0812 07:11:01.587462 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":135,"threads_started":1}
I20260812 07:11:01.588449 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.571747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45693 (local address 127.0.0.1:54750)
0812 07:11:01.572458 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.572508 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:01.573554 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:01.574386 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:01.574392 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:01.574917 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:01.575201 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:01.575218 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.576508 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.576511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:01.577321 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:01.577331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.578160 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.578164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:01.578294 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:01.579193 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:01.579213 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:01.581531 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:01.584483 (+ 2952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:01.584498 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:01.584514 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:01.584990 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:01.585535 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:01.585542 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:01.585547 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:01.585765 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:01.586592 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:01.586600 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:01.586861 (+ 261us) client_negotiation.cc:769] Sending connection context
0812 07:11:01.587095 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 07:11:01.587620 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":87,"thread_start_us":228,"threads_started":1}
I20260812 07:11:01.591886 15614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:11:01.593050 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.592687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:35249 (local address 127.0.0.1:38480)
0812 07:11:01.592841 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.592945 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 07:11:01.593300 15556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
I20260812 07:11:01.594581 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.594162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:35249 (local address 127.0.0.1:38494)
0812 07:11:01.594397 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.594486 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260812 07:11:01.594914 15614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:35249: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab.
Entry for principal kudu/127.31.156.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab.
I20260812 07:11:01.604529 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.125:35249
--webserver_interface=127.31.156.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab
--principal=kudu/127.31.156.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:45693,127.31.156.125:35249,127.31.156.124:33913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:11:01.644264 15614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } attempt: 1
I20260812 07:11:01.645431 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.645028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:35249 (local address 127.0.0.1:38508)
0812 07:11:01.645264 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.645341 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260812 07:11:01.646546 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.646242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:35249 (local address 127.0.0.1:38516)
0812 07:11:01.646409 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.646468 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260812 07:11:01.646754 15614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:35249: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
I20260812 07:11:01.715004 15614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } attempt: 2
I20260812 07:11:01.716650 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.715914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:35249 (local address 127.0.0.1:38526)
0812 07:11:01.716424 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.716533 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 07:11:01.717849 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.717569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:35249 (local address 127.0.0.1:38542)
0812 07:11:01.717698 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.717767 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 07:11:01.718040 15614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:35249: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
W20260812 07:11:01.745920 15622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:01.746330 15622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:01.746423 15622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:01.750952 15622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:11:01.751091 15622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:01.751128 15622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:01.751178 15622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:11:01.751216 15622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:11:01.757194 15622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:45693,127.31.156.125:35249,127.31.156.124:33913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab
--principal=kudu/127.31.156.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.125:35249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.156.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:01.758539 15622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:01.759756 15622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:01.770439 15628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:01.770439 15630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:01.770439 15627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:01.771004 15622 server_base.cc:1034] running on GCE node
I20260812 07:11:01.771947 15622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:01.773403 15622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:01.774641 15622 hybrid_clock.cc:648] HybridClock initialized: now 1786518661774608 us; error 58 us; skew 500 ppm
Aug 12 07:11:01 dist-test-slave-csg5 krb5kdc[15522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:01.778245 15622 init.cc:377] Logged in from keytab as kudu/127.31.156.125@KRBTEST.COM (short username kudu)
I20260812 07:11:01.779585 15622 webserver.cc:460] Webserver started at http://127.31.156.125:43369/ using document root <none> and password file <none>
I20260812 07:11:01.780283 15622 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:01.780361 15622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:01.780557 15622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:01.783164 15622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/data/instance:
uuid: "1bbb770055124cacb5f0b11854665e09"
format_stamp: "Formatted at 2026-08-12 07:11:01 on dist-test-slave-csg5"
server_key: "8e5d4b72905b907abea370c76acb78d5"
server_key_iv: "64db30a965d5c7c7bc05995e580d94f1"
server_key_version: "encryptionkey@0"
I20260812 07:11:01.783825 15622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/wal/instance:
uuid: "1bbb770055124cacb5f0b11854665e09"
format_stamp: "Formatted at 2026-08-12 07:11:01 on dist-test-slave-csg5"
server_key: "8e5d4b72905b907abea370c76acb78d5"
server_key_iv: "64db30a965d5c7c7bc05995e580d94f1"
server_key_version: "encryptionkey@0"
I20260812 07:11:01.788406 15622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:11:01.792183 15638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:01.794476 15622 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260812 07:11:01.794742 15622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/wal
uuid: "1bbb770055124cacb5f0b11854665e09"
format_stamp: "Formatted at 2026-08-12 07:11:01 on dist-test-slave-csg5"
server_key: "8e5d4b72905b907abea370c76acb78d5"
server_key_iv: "64db30a965d5c7c7bc05995e580d94f1"
server_key_version: "encryptionkey@0"
I20260812 07:11:01.794929 15622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:01.809351 15614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } attempt: 3
I20260812 07:11:01.810374 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.810058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:35249 (local address 127.0.0.1:38544)
0812 07:11:01.810201 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.810287 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 07:11:01.811189 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.810947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:35249 (local address 127.0.0.1:38550)
0812 07:11:01.811064 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.811120 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:11:01.811342 15614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:35249: Network error: Client connection negotiation failed: client connection to 127.31.156.125:35249: connect: Connection refused (error 111)
I20260812 07:11:01.824054 15622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:01.828763 15622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:01.829046 15622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:01.838398 15622 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.125:35249
I20260812 07:11:01.838806 15699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.125:35249 every 8 connection(s)
I20260812 07:11:01.839764 15622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
I20260812 07:11:01.843277 15700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:01.845309 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 15622
I20260812 07:11:01.845463 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-1/wal/instance
I20260812 07:11:01.845788 32369 external_mini_cluster.cc:1428] Setting key a4776158ba71ba5094895aed40e152ff
I20260812 07:11:01.849509 15700 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.156.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.124@KRBTEST.COM" created.
Aug 12 07:11:01 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:01.868230 15618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.852271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54764 (local address 127.31.156.126:45693)
0812 07:11:01.852445 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:01.852449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:01.854022 (+ 1573us) server_negotiation.cc:407] Connection header received
0812 07:11:01.855250 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:01.855255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:01.855325 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:01.855439 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:01.856828 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.857449 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.858991 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.859249 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.859378 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:01.863556 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:01.863560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:01.863562 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:01.863594 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:01.865320 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:01.866129 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:01.866134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:01.866135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:01.866234 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:01.866685 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:01.866690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:01.866692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:01.866962 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:01.867106 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:01.867853 (+ 747us) server_negotiation.cc:299] Negotiation successful
0812 07:11:01.868021 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:11:01.869288 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.852826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45693 (local address 127.0.0.1:54764)
0812 07:11:01.853554 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.853682 (+ 128us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:01.854976 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:01.855649 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:01.855657 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:01.856048 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:01.856647 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:01.856663 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.857633 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.857637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:01.858840 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:01.858855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.859516 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.859521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:01.859728 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:01.860819 (+ 1091us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:01.860847 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:01.863289 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:01.865596 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:01.865605 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:01.865635 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:01.865985 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:01.866351 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:01.866354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:01.866356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:01.866541 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:01.867107 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:01.867116 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:01.867445 (+ 329us) client_negotiation.cc:769] Sending connection context
0812 07:11:01.867660 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 07:11:01.868174 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":195,"threads_started":1}
I20260812 07:11:01.871905 15700 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:11:01 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
I20260812 07:11:01.884495 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.872968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:35249 (local address 127.0.0.1:38564)
0812 07:11:01.873220 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.873242 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:01.873391 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:01.874566 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:01.874569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:01.874833 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:01.874930 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:01.874936 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.875999 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.876002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:01.876529 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:01.876536 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.877251 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.877254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:01.877343 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:01.877879 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:01.877892 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:01.879715 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:01.882428 (+ 2713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:01.882432 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:01.882434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:01.882757 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:01.883151 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:01.883155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:01.883157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:01.883255 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:01.883853 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:01.883856 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:01.883955 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 07:11:01.884047 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 07:11:01.884191 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260812 07:11:01.884529 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.873464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38564 (local address 127.31.156.125:35249)
0812 07:11:01.873947 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:01.873964 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:01.873991 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:11:01.874048 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:01.874053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:01.874367 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:01.874595 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:01.875061 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.875854 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.876651 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.877112 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.877252 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:01.879904 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:01.879917 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:01.879933 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:01.879956 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:01.882267 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:01.882945 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:01.882960 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:01.882961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:01.883047 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:01.883362 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:01.883365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:01.883367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:01.883684 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:01.883868 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:01.884180 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 07:11:01.884336 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":158,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab.
Entry for principal kudu/127.31.156.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab.
I20260812 07:11:01.887347 15700 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:11:01.888305 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.887956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41542)
0812 07:11:01.888124 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.888225 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 07:11:01.888237 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.124:33913
--webserver_interface=127.31.156.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab
--principal=kudu/127.31.156.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:45693,127.31.156.125:35249,127.31.156.124:33913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:01.888708 15642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
I20260812 07:11:01.889909 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.889603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41552)
0812 07:11:01.889780 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.889837 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260812 07:11:01.890177 15700 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:33913: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
I20260812 07:11:01.939630 15700 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } attempt: 1
I20260812 07:11:01.940807 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.940313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41566)
0812 07:11:01.940606 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.940692 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 07:11:01.941812 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.941530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41570)
0812 07:11:01.941657 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.941718 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 07:11:01.942015 15700 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:33913: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
I20260812 07:11:01.954555 15614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } attempt: 4
Aug 12 07:11:01 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
I20260812 07:11:01.967160 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.955479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:35249 (local address 127.0.0.1:38570)
0812 07:11:01.955764 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.955780 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:01.955889 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:01.956212 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:01.956215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:01.956432 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:01.956566 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:01.956573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.957788 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.957792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:01.958641 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:01.958698 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.959408 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.959411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:01.959514 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:01.960203 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:01.960217 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:01.961867 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:01.964675 (+ 2808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:01.964678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:01.964680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:01.964983 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:01.965655 (+ 672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:01.965659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:01.965662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:01.965794 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:01.966380 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:01.966382 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:01.966518 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 07:11:01.966619 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 07:11:01.966976 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 07:11:01.967319 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.955568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38570 (local address 127.31.156.125:35249)
0812 07:11:01.955732 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:01.955737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:01.955865 (+ 128us) server_negotiation.cc:407] Connection header received
0812 07:11:01.955990 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:01.955993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:01.956051 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:01.956161 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:01.956762 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.957604 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.958821 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:01.959191 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:01.959308 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:01.962062 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:01.962067 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:01.962070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:01.962105 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:01.964458 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:01.965265 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:01.965269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:01.965353 (+ 84us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:01.965505 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:01.965927 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:01.965931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:01.965933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:01.966237 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:01.966359 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:01.966945 (+ 586us) server_negotiation.cc:299] Negotiation successful
0812 07:11:01.967122 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:11:01.968786 15614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:11:01.969944 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.969566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41580)
0812 07:11:01.969750 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.969829 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 07:11:01.971058 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:01.970763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41584)
0812 07:11:01.970903 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:01.970973 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:11:01.971258 15614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:33913: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
I20260812 07:11:02.010245 15700 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } attempt: 2
I20260812 07:11:02.011358 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.011001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41592)
0812 07:11:02.011171 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.011268 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 07:11:02.012540 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.012122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41606)
0812 07:11:02.012393 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.012463 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196}
W20260812 07:11:02.012741 15700 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:33913: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
I20260812 07:11:02.030458 15614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } attempt: 1
I20260812 07:11:02.031608 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.031152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41610)
0812 07:11:02.031416 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.031497 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 07:11:02.032639 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.032325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41616)
0812 07:11:02.032470 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.032550 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 07:11:02.033020 15614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:33913: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
W20260812 07:11:02.036958 15707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:02.037364 15707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:02.037478 15707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:02.043242 15707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:11:02.043370 15707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:02.043413 15707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:02.043469 15707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:11:02.043525 15707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:11:02.050514 15707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:45693,127.31.156.125:35249,127.31.156.124:33913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab
--principal=kudu/127.31.156.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.124:33913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.156.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:02.051964 15707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:02.053195 15707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:02.063853 15712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:02.063866 15715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:02.063927 15707 server_base.cc:1034] running on GCE node
W20260812 07:11:02.063854 15713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:02.064888 15707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:02.066983 15707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:02.068230 15707 hybrid_clock.cc:648] HybridClock initialized: now 1786518662068259 us; error 80 us; skew 500 ppm
Aug 12 07:11:02 dist-test-slave-csg5 krb5kdc[15522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518662, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:02.071663 15707 init.cc:377] Logged in from keytab as kudu/127.31.156.124@KRBTEST.COM (short username kudu)
I20260812 07:11:02.072750 15707 webserver.cc:460] Webserver started at http://127.31.156.124:37015/ using document root <none> and password file <none>
I20260812 07:11:02.073341 15707 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:02.073413 15707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:02.073599 15707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:02.076553 15707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/data/instance:
uuid: "006f4d4d330547d690db8c342834b4fe"
format_stamp: "Formatted at 2026-08-12 07:11:02 on dist-test-slave-csg5"
server_key: "3049a9e955eef7649fd0bdc5a6d30ea5"
server_key_iv: "f7f21ac6f990963e373457114be457fc"
server_key_version: "encryptionkey@0"
I20260812 07:11:02.077198 15707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/wal/instance:
uuid: "006f4d4d330547d690db8c342834b4fe"
format_stamp: "Formatted at 2026-08-12 07:11:02 on dist-test-slave-csg5"
server_key: "3049a9e955eef7649fd0bdc5a6d30ea5"
server_key_iv: "f7f21ac6f990963e373457114be457fc"
server_key_version: "encryptionkey@0"
I20260812 07:11:02.082093 15707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:11:02.085827 15724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:02.087561 15707 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260812 07:11:02.087818 15707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/wal
uuid: "006f4d4d330547d690db8c342834b4fe"
format_stamp: "Formatted at 2026-08-12 07:11:02 on dist-test-slave-csg5"
server_key: "3049a9e955eef7649fd0bdc5a6d30ea5"
server_key_iv: "f7f21ac6f990963e373457114be457fc"
server_key_version: "encryptionkey@0"
I20260812 07:11:02.087987 15707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:02.100337 15614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } attempt: 2
I20260812 07:11:02.101562 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.100992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41630)
0812 07:11:02.101214 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.101425 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 07:11:02.102619 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.102295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41640)
0812 07:11:02.102460 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.102533 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 07:11:02.102881 15614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:33913: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
I20260812 07:11:02.103953 15700 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } attempt: 3
I20260812 07:11:02.105176 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.104873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41642)
0812 07:11:02.105018 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.105088 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 07:11:02.105752 15707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:02.106218 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.105938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41648)
0812 07:11:02.106097 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.106153 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 07:11:02.106351 15700 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:33913: Network error: Client connection negotiation failed: client connection to 127.31.156.124:33913: connect: Connection refused (error 111)
I20260812 07:11:02.109722 15707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:02.109982 15707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:02.119467 15707 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.124:33913
I20260812 07:11:02.119907 15779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.124:33913 every 8 connection(s)
I20260812 07:11:02.120915 15707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
I20260812 07:11:02.124325 15780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:02.128723 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 15707
I20260812 07:11:02.128871 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518468557119-32369-0/minicluster-data/master-2/wal/instance
I20260812 07:11:02.129197 32369 external_mini_cluster.cc:1428] Setting key 1a6383c37fc4dd4eb5fa97ef8cf9248f
I20260812 07:11:02.130113 15780 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:11:02 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518662, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
Aug 12 07:11:02 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
I20260812 07:11:02.146062 15618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.132542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54766 (local address 127.31.156.126:45693)
0812 07:11:02.132699 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:02.132704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:02.133097 (+ 393us) server_negotiation.cc:407] Connection header received
0812 07:11:02.133556 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:02.133560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:02.133626 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:02.133722 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:02.134627 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.135191 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.136241 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.136470 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.136584 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.140002 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:02.140010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:02.140017 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:02.140060 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:02.142337 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.143219 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.143223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.143225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.143341 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.143852 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.143856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.143858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.144131 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:02.144234 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:02.145739 (+ 1505us) server_negotiation.cc:299] Negotiation successful
0812 07:11:02.145889 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:11:02.146520 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.131095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41650 (local address 127.31.156.124:33913)
0812 07:11:02.131632 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:02.131649 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:02.131702 (+ 53us) server_negotiation.cc:407] Connection header received
0812 07:11:02.132919 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:02.132945 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:02.133338 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:02.133611 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:02.134686 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.135682 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.136639 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.137045 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.137187 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.140557 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:02.140565 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:02.140579 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:02.140605 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:02.143097 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.143728 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.143736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.143737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.143865 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.144436 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.144439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.144440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.144725 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:02.144885 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:02.145381 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 07:11:02.145590 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"spinlock_wait_cycles":8960,"thread_start_us":150,"threads_started":1}
I20260812 07:11:02.146525 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.132431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45693 (local address 127.0.0.1:54766)
0812 07:11:02.132953 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.132998 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:02.133329 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:02.133780 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:02.133786 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:02.134183 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:02.134456 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:02.134469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.135310 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.135314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:02.136105 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:02.136114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.136635 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.136639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:02.136838 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.137494 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:02.137512 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:02.139768 (+ 2256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:02.142491 (+ 2723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:02.142503 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:02.142514 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:02.143096 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:02.143446 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:02.143449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:02.143452 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:02.143669 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:02.144280 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:02.144289 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:02.144658 (+ 369us) client_negotiation.cc:769] Sending connection context
0812 07:11:02.144969 (+ 311us) client_negotiation.cc:240] Negotiation successful
0812 07:11:02.145394 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"spinlock_wait_cycles":9088,"thread_start_us":220,"threads_started":1}
I20260812 07:11:02.148325 15780 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:11:02 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518662, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
Aug 12 07:11:02 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:02.160521 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.149113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38576 (local address 127.31.156.125:35249)
0812 07:11:02.149259 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:02.149263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:02.149713 (+ 450us) server_negotiation.cc:407] Connection header received
0812 07:11:02.149879 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:02.149883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:02.149965 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:02.150052 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:02.150617 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.151289 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.152315 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.152728 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.152847 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.155727 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:02.155732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:02.155735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:02.155778 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:02.157777 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.158565 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.158568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.158570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.158835 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.159224 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.159229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.159231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.159713 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:02.159915 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:02.160251 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 07:11:02.160378 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:11:02.160545 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.148979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:35249 (local address 127.0.0.1:38576)
0812 07:11:02.149605 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.149631 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:02.149750 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:02.150106 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:02.150108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:02.150369 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:02.150481 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:02.150487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.151412 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.151416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:02.152170 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:02.152180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.152889 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.152892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:02.153004 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.153699 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:02.153713 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:02.155512 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:02.158093 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:02.158100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:02.158105 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:02.158413 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:02.158961 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:02.158965 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:02.158967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:02.159080 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:02.159920 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:02.159925 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:02.160038 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 07:11:02.160159 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 07:11:02.160347 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528}
I20260812 07:11:02.161619 15618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.149606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.31.156.126:45693)
0812 07:11:02.149782 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:02.149786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:02.149798 (+ 12us) server_negotiation.cc:407] Connection header received
0812 07:11:02.149840 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:02.149842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:02.149883 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:02.149954 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:02.150940 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.151615 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.152869 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.153245 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.153345 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.156764 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:02.156768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:02.156770 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:02.156800 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:02.158999 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.159698 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.159701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.159702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.159796 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.160246 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.160250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.160251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.160537 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:02.160826 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:02.161253 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 07:11:02.161401 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 07:11:02.161682 15780 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:11:02 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518662, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
Aug 12 07:11:02 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
I20260812 07:11:02.175781 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.162271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41658)
0812 07:11:02.162806 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.162829 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:02.162954 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:02.163417 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:02.163421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:02.163672 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:02.163810 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:02.163817 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.166160 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.166164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:02.167055 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:02.167069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.167721 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.167725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:02.167793 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.168400 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:02.168423 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:02.171303 (+ 2880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:02.173495 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:02.173499 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:02.173502 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:02.173827 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:02.174529 (+ 702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:02.174535 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:02.174541 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:02.174742 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:02.175235 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:02.175238 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:02.175373 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 07:11:02.175485 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 07:11:02.175627 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260812 07:11:02.176263 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.162804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41658 (local address 127.31.156.124:33913)
0812 07:11:02.162930 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:02.162934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:02.162948 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:11:02.163034 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:02.163039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:02.163075 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:02.163249 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:02.164035 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.165952 (+ 1917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.167220 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.167578 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.167697 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.171521 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:02.171526 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:02.171528 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:02.171561 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:02.173361 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.174135 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.174141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.174144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.174368 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.174886 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.174889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.174891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.175112 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:02.175208 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:02.175601 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 07:11:02.175901 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:11:02.178610 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.163303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38586 (local address 127.31.156.125:35249)
0812 07:11:02.163450 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:02.163456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:02.163613 (+ 157us) server_negotiation.cc:407] Connection header received
0812 07:11:02.163789 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:02.163793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:02.163847 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:02.163954 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:02.166170 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.167020 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.168122 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.168359 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.168484 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.172724 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:02.172731 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:02.172736 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:02.172781 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:02.175237 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.176505 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.176509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.176511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.176620 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.177627 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.177631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.177633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.177888 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:02.178218 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:02.178299 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 07:11:02.178434 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:11:02.181429 15780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe: Bootstrap starting.
I20260812 07:11:02.183178 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:11:02.184304 15780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:02.185257 15780 log.cc:824] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:02.188134 15780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe: No bootstrap required, opened a new log
I20260812 07:11:02.193325 15780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } }
I20260812 07:11:02.193881 15780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:02.193966 15780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 006f4d4d330547d690db8c342834b4fe, State: Initialized, Role: FOLLOWER
I20260812 07:11:02.194594 15780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } }
I20260812 07:11:02.195290 15794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } } }
I20260812 07:11:02.195586 15794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:11:02.196341 15780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:11:02.198514 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.185632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41674 (local address 127.31.156.124:33913)
0812 07:11:02.186342 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:02.186347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:02.186493 (+ 146us) server_negotiation.cc:407] Connection header received
0812 07:11:02.186586 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:02.186589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:02.186635 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:02.186737 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:02.187954 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.188786 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.190317 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.190714 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.190799 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.193456 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:02.193461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:02.193464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:02.193499 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:02.195833 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.196440 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.196444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.196445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.196546 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.196915 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.196919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.196921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.197205 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:02.198008 (+ 803us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:02.198091 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 07:11:02.198225 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628}
I20260812 07:11:02.198879 15618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.186326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.31.156.126:45693)
0812 07:11:02.186443 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:02.186447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:02.186481 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:11:02.186536 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:02.186539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:02.186595 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:02.186738 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:02.187955 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.188776 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.190326 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.190709 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.190793 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.193498 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:02.193503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:02.193506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:02.193536 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:02.195964 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.197343 (+ 1379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.197346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.197348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.197478 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.197815 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.197819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.197821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.198066 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:02.198220 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:02.198527 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 07:11:02.198700 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:11:02.202034 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.186062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38590 (local address 127.31.156.125:35249)
0812 07:11:02.186196 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:02.186201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:02.186901 (+ 700us) server_negotiation.cc:407] Connection header received
0812 07:11:02.186951 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:02.186954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:02.187002 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:02.187074 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:02.188090 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.188840 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.190274 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.190607 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.190782 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.193553 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:02.193557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:02.193560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:02.193598 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:02.196851 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.200348 (+ 3497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.200355 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.200356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.200469 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.200829 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.200833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.200834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.201049 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:02.201225 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:02.201657 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 07:11:02.201809 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 07:11:02.205411 15807 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:11:02.205542 15807 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:11:02.201122 15707 master_runner.cc:186] Error getting master registration for 127.31.156.126:45693: OK, instance_id { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" instance_seqno: 1786518661556331 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:11:02.206473 15707 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:11:02.206614 15614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } attempt: 3
Aug 12 07:11:02 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
I20260812 07:11:02.219516 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.207163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41680)
0812 07:11:02.207470 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.207490 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:02.207585 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:02.208262 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:02.208265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:02.208477 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:02.208767 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:02.208799 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.210188 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.210193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:02.211133 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:02.211148 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.211706 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.211710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:02.211818 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.212661 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:02.212678 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:02.214705 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:02.217608 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:02.217611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:02.217612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:02.217880 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:02.218270 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:02.218273 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:02.218274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:02.218370 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:02.218976 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:02.218978 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:02.219098 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 07:11:02.219203 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 07:11:02.219367 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211}
I20260812 07:11:02.219666 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.207777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41680 (local address 127.31.156.124:33913)
0812 07:11:02.207911 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:02.207916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:02.207931 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:11:02.207973 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:02.207976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:02.208025 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:02.208113 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:02.209311 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.209883 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.211294 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.211561 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.211697 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.214888 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:02.214893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:02.214896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:02.214933 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:02.217426 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.218020 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.218024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.218026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.218139 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.218497 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.218501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.218503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.218811 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:02.218953 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:02.219295 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 07:11:02.219457 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:11:02.223910 15614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7: Bootstrap starting.
I20260812 07:11:02.227443 15614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:02.228634 15614 log.cc:824] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:02.231752 15614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7: No bootstrap required, opened a new log
I20260812 07:11:02.237488 15614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } }
I20260812 07:11:02.237783 15614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:02.237896 15614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 743b733f34be41aeb7ea6a57a47e87a7, State: Initialized, Role: FOLLOWER
I20260812 07:11:02.238492 15614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } }
I20260812 07:11:02.239225 15810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } } }
I20260812 07:11:02.239401 15810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:11:02.240221 15614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:11:02.247660 15822 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:11:02.247963 15822 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:11:02.249524 15700 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } attempt: 4
I20260812 07:11:02.249811 15810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:11:02.250016 15810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } }
I20260812 07:11:02.251479 15810 leader_election.cc:288] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bbb770055124cacb5f0b11854665e09 (127.31.156.125:35249), 006f4d4d330547d690db8c342834b4fe (127.31.156.124:33913)
I20260812 07:11:02.252597 15753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "743b733f34be41aeb7ea6a57a47e87a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "006f4d4d330547d690db8c342834b4fe" is_pre_election: true
I20260812 07:11:02.253494 15753 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 743b733f34be41aeb7ea6a57a47e87a7 in term 0.
I20260812 07:11:02.254230 15559 leader_election.cc:302] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 006f4d4d330547d690db8c342834b4fe, 743b733f34be41aeb7ea6a57a47e87a7; no voters:
I20260812 07:11:02.249467 15537 master_runner.cc:186] Error getting master registration for 127.31.156.125:35249: OK, instance_id { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" instance_seqno: 1786518661835584 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:11:02.255651 15537 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:11:02.255878 15810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:11:02.255975 15810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:11:02.256021 15810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:02.257922 15810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } }
I20260812 07:11:02.258438 15810 leader_election.cc:288] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [CANDIDATE]: Term 1 election: Requested vote from peers 1bbb770055124cacb5f0b11854665e09 (127.31.156.125:35249), 006f4d4d330547d690db8c342834b4fe (127.31.156.124:33913)
I20260812 07:11:02.260284 15753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "743b733f34be41aeb7ea6a57a47e87a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "006f4d4d330547d690db8c342834b4fe"
I20260812 07:11:02.260450 15753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [term 0 FOLLOWER]: Advancing to term 1
Aug 12 07:11:02 dist-test-slave-csg5 krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518661, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
I20260812 07:11:02.264161 15753 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 743b733f34be41aeb7ea6a57a47e87a7 in term 1.
I20260812 07:11:02.268031 15559 leader_election.cc:302] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 006f4d4d330547d690db8c342834b4fe, 743b733f34be41aeb7ea6a57a47e87a7; no voters:
I20260812 07:11:02.270471 15810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:11:02.270794 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.255233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:33913 (local address 127.0.0.1:41686)
0812 07:11:02.255547 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:02.255563 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:02.255663 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:02.256308 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:02.256311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:02.256539 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:02.256650 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:02.256656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.257781 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.257785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:02.258685 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:02.258697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.260712 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.260716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:02.260820 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.261440 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:02.261458 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:02.263415 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:02.266770 (+ 3355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:02.266775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:02.266777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:02.267549 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:02.268048 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:02.268052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:02.268054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:02.268160 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:02.269960 (+ 1800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:02.269964 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:02.270125 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 07:11:02.270240 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 07:11:02.270462 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260812 07:11:02.271126 15810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [term 1 LEADER]: Becoming Leader. State: Replica: 743b733f34be41aeb7ea6a57a47e87a7, State: Running, Role: LEADER
I20260812 07:11:02.271916 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:02.255844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41686 (local address 127.31.156.124:33913)
0812 07:11:02.255972 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:02.255976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:02.255990 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:11:02.256034 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:02.256037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:02.256083 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:02.256158 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:02.256795 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.257637 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.259969 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:02.260348 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:02.260464 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:02.263644 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:02.263649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:02.263652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:02.263688 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:02.266314 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.267727 (+ 1413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.267732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.267735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.267921 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:02.269032 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:02.269039 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:02.269041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:02.269716 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:02.269890 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:02.271532 (+ 1642us) server_negotiation.cc:299] Negotiation successful
0812 07:11:02.271707 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:11:02.272229 15810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } }
I20260812 07:11:02.278527 15700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09: Bootstrap starting.
I20260812 07:11:02.282189 15824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 743b733f34be41aeb7ea6a57a47e87a7. Latest consensus state: current_term: 1 leader_uuid: "743b733f34be41aeb7ea6a57a47e87a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } } }
I20260812 07:11:02.282639 15824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:02.283942 15826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:11:02.287217 15700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:02.289178 15700 log.cc:824] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:02.290812 15826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:11:02.300343 15700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09: No bootstrap required, opened a new log
W20260812 07:11:02.302534 15562 tablet.cc:2369] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:11:02.306824 15700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } }
I20260812 07:11:02.307514 15700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:02.307628 15700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bbb770055124cacb5f0b11854665e09, State: Initialized, Role: FOLLOWER
I20260812 07:11:02.308357 15700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } }
I20260812 07:11:02.310601 15753 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [term 1 FOLLOWER]: Refusing update from remote peer 743b733f34be41aeb7ea6a57a47e87a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:02.311169 15829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } } }
I20260812 07:11:02.311750 15810 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:02.311805 15829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:11:02.314131 15671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "743b733f34be41aeb7ea6a57a47e87a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bbb770055124cacb5f0b11854665e09" is_pre_election: true
I20260812 07:11:02.314489 15670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "743b733f34be41aeb7ea6a57a47e87a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bbb770055124cacb5f0b11854665e09"
I20260812 07:11:02.317246 15669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:02.319080 15669 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [term 1 FOLLOWER]: Refusing update from remote peer 743b733f34be41aeb7ea6a57a47e87a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:02.319716 15700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:11:02.320103 15810 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:02.321053 15794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 743b733f34be41aeb7ea6a57a47e87a7. Latest consensus state: current_term: 1 leader_uuid: "743b733f34be41aeb7ea6a57a47e87a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } } }
I20260812 07:11:02.321213 15794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:11:02.326110 15810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "743b733f34be41aeb7ea6a57a47e87a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } } }
I20260812 07:11:02.326285 15810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:02.326432 15828 mvcc.cc:205] Tried to move back new op lower bound from 7317580440807276544 to 7317580440685068288. Current Snapshot: MvccSnapshot[applied={T|T < 7317580440807276544}]
I20260812 07:11:02.326824 15794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "743b733f34be41aeb7ea6a57a47e87a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } } }
I20260812 07:11:02.326969 15794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:11:02.326824 15830 mvcc.cc:205] Tried to move back new op lower bound from 7317580440807276544 to 7317580440685068288. Current Snapshot: MvccSnapshot[applied={T|T < 7317580440807276544}]
I20260812 07:11:02.327941 15829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 743b733f34be41aeb7ea6a57a47e87a7. Latest consensus state: current_term: 1 leader_uuid: "743b733f34be41aeb7ea6a57a47e87a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } } }
I20260812 07:11:02.328073 15829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:11:02.331017 15826 catalog_manager.cc:1347] Generated new cluster ID: 42566271481a410095491c96451de1d7
I20260812 07:11:02.331108 15826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260812 07:11:02.331473 15644 tablet.cc:2369] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:11:02.339691 15824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "743b733f34be41aeb7ea6a57a47e87a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } } }
I20260812 07:11:02.339890 15824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7 [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:02.341132 15829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "743b733f34be41aeb7ea6a57a47e87a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "743b733f34be41aeb7ea6a57a47e87a7" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45693 } } peers { permanent_uuid: "1bbb770055124cacb5f0b11854665e09" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 35249 } } peers { permanent_uuid: "006f4d4d330547d690db8c342834b4fe" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 33913 } } }
I20260812 07:11:02.341300 15829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 07:11:02.344964 15847 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:11:02.345829 15847 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:11:02.358305 15843 mvcc.cc:205] Tried to move back new op lower bound from 7317580440807276544 to 7317580440685068288. Current Snapshot: MvccSnapshot[applied={T|T < 7317580440807276544}]
I20260812 07:11:02.386181 15826 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:11:02.387746 15826 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:11:02.398278 15826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 743b733f34be41aeb7ea6a57a47e87a7: Generated new TSK 0
I20260812 07:11:02.399376 15826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:11:03.207136 15807 catalog_manager.cc:1259] Loaded cluster ID: 42566271481a410095491c96451de1d7
I20260812 07:11:03.207244 15807 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe: loading cluster ID for follower catalog manager: success
I20260812 07:11:03.209196 15807 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe: acquiring CA information for follower catalog manager: success
I20260812 07:11:03.210692 15807 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 006f4d4d330547d690db8c342834b4fe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:11:03.347517 15847 catalog_manager.cc:1259] Loaded cluster ID: 42566271481a410095491c96451de1d7
I20260812 07:11:03.347615 15847 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09: loading cluster ID for follower catalog manager: success
I20260812 07:11:03.349450 15847 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09: acquiring CA information for follower catalog manager: success
I20260812 07:11:03.350795 15847 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1bbb770055124cacb5f0b11854665e09: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:11:03.486094 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 15537
I20260812 07:11:03.498965 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 15622
I20260812 07:11:03.511030 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 15707
2026-08-12T07:11:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3601 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:11:03 dist-test-slave-csg5 krb5kdc[15858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:11:03 dist-test-slave-csg5 krb5kdc[15858](info): set up 2 sockets
Aug 12 07:11:03 dist-test-slave-csg5 krb5kdc[15858](info): commencing operation
krb5kdc: starting...
W20260812 07:11:04.752043 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:11:04 dist-test-slave-csg5 krb5kdc[15858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:11:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:11:04.935897 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:38173
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:38173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:05.064929 15872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:05.065341 15872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:05.065430 15872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:05.070770 15872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:11:05.071086 15872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:05.071130 15872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:05.071172 15872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:11:05.071208 15872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:11:05.079174 15872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:38173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:38173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:05.080575 15872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:05.081801 15872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:11:05.092190 15872 server_base.cc:1034] running on GCE node
W20260812 07:11:05.092139 15880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:05.092139 15878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:05.092144 15882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:05.093274 15872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:05.094982 15872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:05.096311 15872 hybrid_clock.cc:648] HybridClock initialized: now 1786518665096287 us; error 43 us; skew 500 ppm
Aug 12 07:11:05 dist-test-slave-csg5 krb5kdc[15858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518665, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:05.105660 15872 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:11:05.107003 15872 webserver.cc:460] Webserver started at http://127.31.156.126:45293/ using document root <none> and password file <none>
I20260812 07:11:05.107695 15872 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:05.107813 15872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:05.108114 15872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:05.110497 15872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "26b9342228304e2194241ef4751c3974"
format_stamp: "Formatted at 2026-08-12 07:11:05 on dist-test-slave-csg5"
server_key: "550a33d722376a4b765959674202f430"
server_key_iv: "f1675ed20c3d66e79105a8429020268c"
server_key_version: "encryptionkey@0"
I20260812 07:11:05.111243 15872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "26b9342228304e2194241ef4751c3974"
format_stamp: "Formatted at 2026-08-12 07:11:05 on dist-test-slave-csg5"
server_key: "550a33d722376a4b765959674202f430"
server_key_iv: "f1675ed20c3d66e79105a8429020268c"
server_key_version: "encryptionkey@0"
I20260812 07:11:05.115816 15872 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:11:05.118984 15889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:05.120762 15872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:11:05.120913 15872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "26b9342228304e2194241ef4751c3974"
format_stamp: "Formatted at 2026-08-12 07:11:05 on dist-test-slave-csg5"
server_key: "550a33d722376a4b765959674202f430"
server_key_iv: "f1675ed20c3d66e79105a8429020268c"
server_key_version: "encryptionkey@0"
I20260812 07:11:05.121043 15872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:05.138912 15872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:05.143282 15872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:05.143489 15872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:05.151607 15872 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:38173
I20260812 07:11:05.151638 15948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:38173 every 8 connection(s)
I20260812 07:11:05.152810 15872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:11:05.156098 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 15872
I20260812 07:11:05.156221 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:11:05.156383 15949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:05.156615 32369 external_mini_cluster.cc:1428] Setting key 7f2019fd081d40615c73734d6828de1a
I20260812 07:11:05.164774 15949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974: Bootstrap starting.
Aug 12 07:11:05 dist-test-slave-csg5 krb5kdc[15858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:05.167924 15949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:05.168995 15949 log.cc:824] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:05.171837 15949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974: No bootstrap required, opened a new log
I20260812 07:11:05.175596 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:05.158167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.31.156.126:38173)
0812 07:11:05.158745 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:05.158766 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:05.158816 (+ 50us) server_negotiation.cc:407] Connection header received
0812 07:11:05.159696 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:05.159724 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:05.160188 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:05.160608 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:05.161458 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:05.163026 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:05.164031 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:05.164781 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:05.164963 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:05.168528 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:05.168536 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:05.168556 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:05.168598 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:05.171540 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:05.172286 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:05.172293 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:05.172309 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:05.172463 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:05.172883 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:05.172886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:05.172888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:05.173535 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:05.173748 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:05.174175 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 07:11:05.174733 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":158,"threads_started":1}
I20260812 07:11:05.176852 15949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b9342228304e2194241ef4751c3974" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 38173 } }
I20260812 07:11:05.177170 15949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:05.177268 15949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26b9342228304e2194241ef4751c3974, State: Initialized, Role: FOLLOWER
I20260812 07:11:05.177875 15949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b9342228304e2194241ef4751c3974" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 38173 } }
I20260812 07:11:05.178078 15949 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:11:05.178153 15949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:11:05.178280 15949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:05.179731 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:11:05.180574 15949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b9342228304e2194241ef4751c3974" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 38173 } }
I20260812 07:11:05.181062 15949 leader_election.cc:302] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26b9342228304e2194241ef4751c3974; no voters:
I20260812 07:11:05.181532 15949 leader_election.cc:288] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:11:05.181797 15954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:11:05.182284 15954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [term 1 LEADER]: Becoming Leader. State: Replica: 26b9342228304e2194241ef4751c3974, State: Running, Role: LEADER
I20260812 07:11:05.182934 15954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b9342228304e2194241ef4751c3974" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 38173 } }
I20260812 07:11:05.183243 15949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:11:05.184985 15957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26b9342228304e2194241ef4751c3974. Latest consensus state: current_term: 1 leader_uuid: "26b9342228304e2194241ef4751c3974" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b9342228304e2194241ef4751c3974" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 38173 } } }
I20260812 07:11:05.185217 15957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:05.185035 15956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26b9342228304e2194241ef4751c3974" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b9342228304e2194241ef4751c3974" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 38173 } } }
I20260812 07:11:05.185501 15956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974 [sys.catalog]: This master's current role is: LEADER
W20260812 07:11:05.188761 15971 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:11:05.188890 15971 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:11:05.188972 15969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:11:05.189714 15969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:11:05.196509 15969 catalog_manager.cc:1347] Generated new cluster ID: 0bba9050a450436985cebd48546d1f43
I20260812 07:11:05.196631 15969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:11:05.203692 15969 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:11:05.205085 15969 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:11:05.214387 15969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26b9342228304e2194241ef4751c3974: Generated new TSK 0
I20260812 07:11:05.215453 15969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:11:05.237630 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 15872
2026-08-12T07:11:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1723 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:11:05 dist-test-slave-csg5 krb5kdc[15979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:11:05 dist-test-slave-csg5 krb5kdc[15979](info): set up 2 sockets
Aug 12 07:11:05 dist-test-slave-csg5 krb5kdc[15979](info): commencing operation
krb5kdc: starting...
W20260812 07:11:06.485044 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:11:06 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:11:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:11:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:11:06.656559 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46033
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:46033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:06.779031 15993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:06.779376 15993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:06.779420 15993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:06.783712 15993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:11:06.783808 15993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:06.783834 15993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:06.783854 15993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:11:06.783870 15993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:11:06.789615 15993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:46033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:06.790912 15993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:06.792078 15993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:06.802141 15999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:06.802206 15993 server_base.cc:1034] running on GCE node
W20260812 07:11:06.802265 16001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:06.802141 15998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:06.803210 15993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:06.804538 15993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:06.805768 15993 hybrid_clock.cc:648] HybridClock initialized: now 1786518666805729 us; error 55 us; skew 500 ppm
Aug 12 07:11:06 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518666, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:06.809195 15993 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:11:06.810299 15993 webserver.cc:460] Webserver started at http://127.31.156.126:34173/ using document root <none> and password file <none>
I20260812 07:11:06.811005 15993 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:06.811108 15993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:06.811373 15993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:06.813634 15993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "ad7107e2ce0e4c749ebf6ab04768b72f"
format_stamp: "Formatted at 2026-08-12 07:11:06 on dist-test-slave-csg5"
server_key: "03a8481afce765a266bd1eb896a37093"
server_key_iv: "0a9b4920c4388dc5ed405bf77fd9ae41"
server_key_version: "encryptionkey@0"
I20260812 07:11:06.814314 15993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "ad7107e2ce0e4c749ebf6ab04768b72f"
format_stamp: "Formatted at 2026-08-12 07:11:06 on dist-test-slave-csg5"
server_key: "03a8481afce765a266bd1eb896a37093"
server_key_iv: "0a9b4920c4388dc5ed405bf77fd9ae41"
server_key_version: "encryptionkey@0"
I20260812 07:11:06.818733 15993 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:11:06.821637 16010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:06.823431 15993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:06.823609 15993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "ad7107e2ce0e4c749ebf6ab04768b72f"
format_stamp: "Formatted at 2026-08-12 07:11:06 on dist-test-slave-csg5"
server_key: "03a8481afce765a266bd1eb896a37093"
server_key_iv: "0a9b4920c4388dc5ed405bf77fd9ae41"
server_key_version: "encryptionkey@0"
I20260812 07:11:06.823755 15993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:06.839313 15993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:06.843017 15993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:06.843289 15993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:06.851634 15993 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:46033
I20260812 07:11:06.851999 16069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:46033 every 8 connection(s)
I20260812 07:11:06.852861 15993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:11:06.855952 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 15993
I20260812 07:11:06.856010 16070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:06.856091 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:11:06.856458 32369 external_mini_cluster.cc:1428] Setting key 29826230d6cd4f884c973492bc895ab9
I20260812 07:11:06.864319 16070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Bootstrap starting.
I20260812 07:11:06.867142 16070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Neither blocks nor log segments found. Creating new log.
Aug 12 07:11:06 dist-test-slave-csg5 krb5kdc[15979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:06.868170 16070 log.cc:824] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:06.870808 16070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: No bootstrap required, opened a new log
I20260812 07:11:06.873947 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:06.858215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36064 (local address 127.31.156.126:46033)
0812 07:11:06.858876 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:06.858900 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:06.858959 (+ 59us) server_negotiation.cc:407] Connection header received
0812 07:11:06.860508 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:06.860532 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:06.860923 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:06.861342 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:06.862241 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:06.863424 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:06.864345 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:06.864903 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:06.865086 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:06.868285 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:06.868295 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:06.868319 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:06.868364 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:06.870494 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:06.871108 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:06.871113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:06.871118 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:06.871244 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:06.871626 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:06.871628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:06.871630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:06.872057 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:06.872214 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:06.872591 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 07:11:06.873004 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":449,"thread_start_us":133,"threads_started":1}
I20260812 07:11:06.875123 16070 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:06.875414 16070 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:06.875515 16070 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad7107e2ce0e4c749ebf6ab04768b72f, State: Initialized, Role: FOLLOWER
I20260812 07:11:06.875988 16070 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:06.876155 16070 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:11:06.876245 16070 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:11:06.876389 16070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:06.877452 16070 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:06.877842 16070 leader_election.cc:302] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad7107e2ce0e4c749ebf6ab04768b72f; no voters:
I20260812 07:11:06.878207 16070 leader_election.cc:288] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:11:06.878449 16075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:11:06.878729 16075 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 1 LEADER]: Becoming Leader. State: Replica: ad7107e2ce0e4c749ebf6ab04768b72f, State: Running, Role: LEADER
I20260812 07:11:06.879127 16075 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:06.879724 16070 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:11:06.880874 16076 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } } }
I20260812 07:11:06.881027 16076 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:06.881417 16077 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad7107e2ce0e4c749ebf6ab04768b72f. Latest consensus state: current_term: 1 leader_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } } }
I20260812 07:11:06.881551 16077 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: This master's current role is: LEADER
W20260812 07:11:06.884740 16090 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:11:06.884821 16090 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:11:06.884907 16084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:11:06.885731 16084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:11:06.892561 16084 catalog_manager.cc:1347] Generated new cluster ID: a390eadcc7f5481ebd5149f3bdc2b774
I20260812 07:11:06.892674 16084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:11:06.905411 16084 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:11:06.906868 16084 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:11:06.921026 16084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Generated new TSK 0
I20260812 07:11:06.922116 16084 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:11:06.960595 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46033
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:07.112450 16097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:07.112797 16097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:07.112847 16097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:07.117327 16097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:07.117426 16097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:07.117492 16097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:11:07.123440 16097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:07.124724 16097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:07.125974 16097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:07.137603 16103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:07.137603 16106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:07.137650 16104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:07.138090 16097 server_base.cc:1034] running on GCE node
I20260812 07:11:07.138944 16097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:07.139775 16097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:07.141028 16097 hybrid_clock.cc:648] HybridClock initialized: now 1786518667140988 us; error 60 us; skew 500 ppm
Aug 12 07:11:07 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518667, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:07.144706 16097 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:11:07.145937 16097 webserver.cc:460] Webserver started at http://127.31.156.65:35421/ using document root <none> and password file <none>
I20260812 07:11:07.146600 16097 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:07.146739 16097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:07.147015 16097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:07.149534 16097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "282de14ccab1062e50213a4bdd19c17b"
server_key_iv: "ac4a9d4b13cc76b7ee3d0b3ac8931354"
server_key_version: "encryptionkey@0"
I20260812 07:11:07.150341 16097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "282de14ccab1062e50213a4bdd19c17b"
server_key_iv: "ac4a9d4b13cc76b7ee3d0b3ac8931354"
server_key_version: "encryptionkey@0"
I20260812 07:11:07.155265 16097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260812 07:11:07.158382 16114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:07.160072 16097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:07.160259 16097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "282de14ccab1062e50213a4bdd19c17b"
server_key_iv: "ac4a9d4b13cc76b7ee3d0b3ac8931354"
server_key_version: "encryptionkey@0"
I20260812 07:11:07.160426 16097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:07.182910 16097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:07.186993 16097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:07.187363 16097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:07.188166 16097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:07.189253 16097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:07.189332 16097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:07.189416 16097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:07.189453 16097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:07.202584 16097 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:34273
I20260812 07:11:07.202687 16237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:34273 every 8 connection(s)
I20260812 07:11:07.203819 16097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:11:07.211763 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 16097
I20260812 07:11:07.211977 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:11:07.212499 32369 external_mini_cluster.cc:1428] Setting key 0207cb66e09b2c047a0b1061f733eb51
Aug 12 07:11:07 dist-test-slave-csg5 krb5kdc[15979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518667, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:07.226567 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.206758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:45313 (local address 127.31.156.126:46033)
0812 07:11:07.206967 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:07.206972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:07.207942 (+ 970us) server_negotiation.cc:407] Connection header received
0812 07:11:07.209768 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:07.209775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:07.209863 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:07.209981 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:07.211939 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.212988 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.215219 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.215647 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.215888 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.220165 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:07.220169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:07.220172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:07.220205 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:07.222112 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:07.223139 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:07.223142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:07.223144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:07.223265 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:07.223988 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:07.223994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:07.223996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:07.224439 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:07.224602 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:07.225829 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0812 07:11:07.226099 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:11:07.227308 16241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.207103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:45313)
0812 07:11:07.207734 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:07.207799 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:07.209336 (+ 1537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:07.210255 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:07.210268 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:07.211009 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:07.211670 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.211689 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.213271 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.213275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.215038 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.215054 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.216112 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.216120 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.216427 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.217524 (+ 1097us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:07.217546 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:07.219838 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:07.222315 (+ 2477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:07.222328 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:07.222357 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:07.222952 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:07.223399 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:07.223405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:07.223408 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:07.223683 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:07.224636 (+ 953us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:07.224646 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:07.225232 (+ 586us) client_negotiation.cc:769] Sending connection context
0812 07:11:07.225515 (+ 283us) client_negotiation.cc:240] Negotiation successful
0812 07:11:07.226029 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":166,"threads_started":1}
I20260812 07:11:07.229669 16239 heartbeater.cc:347] Connected to a master server at 127.31.156.126:46033
I20260812 07:11:07.230189 16239 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
I20260812 07:11:07.231140 16239 heartbeater.cc:511] Master 127.31.156.126:46033 requested a full tablet report, sending...
I20260812 07:11:07.233397 16031 ts_manager.cc:194] Registered new tserver with Master: 89bf44c8f45e49ac8d4eb7129cfdcca2 (127.31.156.65:34273)
I20260812 07:11:07.235510 16031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:45313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:11:07.259709 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46033
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:07.398379 16244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:07.398836 16244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:07.398932 16244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:07.403340 16244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:07.403469 16244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:07.403580 16244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:11:07.409569 16244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:07.410921 16244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:07.412115 16244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:07.424080 16252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:07.424082 16255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:07.424760 16244 server_base.cc:1034] running on GCE node
W20260812 07:11:07.424082 16253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:07.425280 16244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:07.426050 16244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:07.427327 16244 hybrid_clock.cc:648] HybridClock initialized: now 1786518667427265 us; error 67 us; skew 500 ppm
Aug 12 07:11:07 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518667, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:07.431275 16244 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:11:07.432464 16244 webserver.cc:460] Webserver started at http://127.31.156.66:36979/ using document root <none> and password file <none>
I20260812 07:11:07.433214 16244 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:07.433389 16244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:07.433609 16244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:07.436045 16244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "28668e338ddd4aacaacba83a4f86fbb5"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "22597b2c07b93c092763500ce89a25bc"
server_key_iv: "771ee5778821402c8e76df107f505e55"
server_key_version: "encryptionkey@0"
I20260812 07:11:07.436673 16244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "28668e338ddd4aacaacba83a4f86fbb5"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "22597b2c07b93c092763500ce89a25bc"
server_key_iv: "771ee5778821402c8e76df107f505e55"
server_key_version: "encryptionkey@0"
I20260812 07:11:07.441331 16244 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:11:07.444293 16262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:07.445849 16244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:11:07.446013 16244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "28668e338ddd4aacaacba83a4f86fbb5"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "22597b2c07b93c092763500ce89a25bc"
server_key_iv: "771ee5778821402c8e76df107f505e55"
server_key_version: "encryptionkey@0"
I20260812 07:11:07.446159 16244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:07.462268 16244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:07.466203 16244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:07.466517 16244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:07.467314 16244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:07.468288 16244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:07.468356 16244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:07.468405 16244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:07.468466 16244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:07.479698 16244 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:41829
I20260812 07:11:07.479828 16387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:41829 every 8 connection(s)
I20260812 07:11:07.480906 16244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:11:07.490263 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 16244
I20260812 07:11:07.490404 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:11:07.490867 32369 external_mini_cluster.cc:1428] Setting key 087351062d9316230d497a26c2b00f96
Aug 12 07:11:07 dist-test-slave-csg5 krb5kdc[15979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518667, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:07.498692 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.483353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:48257 (local address 127.31.156.126:46033)
0812 07:11:07.483507 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:07.483511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:07.484542 (+ 1031us) server_negotiation.cc:407] Connection header received
0812 07:11:07.486105 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:07.486109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:07.486173 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:07.486273 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:07.487751 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.488343 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.489653 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.489913 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.490033 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.493324 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:07.493328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:07.493330 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:07.493389 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:07.495598 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:07.496476 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:07.496483 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:07.496487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:07.496608 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:07.497035 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:07.497042 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:07.497046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:07.497352 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:07.497478 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:07.498186 (+ 708us) server_negotiation.cc:299] Negotiation successful
0812 07:11:07.498382 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:11:07.499714 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.483787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:48257)
0812 07:11:07.484364 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:07.484416 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:07.485851 (+ 1435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:07.486477 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:07.486485 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:07.487010 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:07.487566 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.487582 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.488508 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.488512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.489494 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.489508 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.490113 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.490118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.490318 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.491108 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:07.491126 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:07.493083 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:07.495790 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:07.495798 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:07.495823 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:07.496347 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:07.496725 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:07.496728 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:07.496729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:07.496925 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:07.497539 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:07.497546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:07.497876 (+ 330us) client_negotiation.cc:769] Sending connection context
0812 07:11:07.498100 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 07:11:07.498568 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":139,"threads_started":1}
I20260812 07:11:07.501348 16388 heartbeater.cc:347] Connected to a master server at 127.31.156.126:46033
I20260812 07:11:07.501693 16388 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:07.502439 16388 heartbeater.cc:511] Master 127.31.156.126:46033 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:11:07.503914 16031 ts_manager.cc:194] Registered new tserver with Master: 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829)
I20260812 07:11:07.504633 16031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:48257
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:11:07.531633 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46033
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:07.658537 16393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:07.658969 16393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:07.659019 16393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:07.663817 16393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:07.663908 16393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:07.663973 16393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:11:07.670064 16393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:07.671428 16393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:07.672572 16393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:07.684330 16398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:07.684414 16401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:07.684556 16393 server_base.cc:1034] running on GCE node
W20260812 07:11:07.684854 16399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:07.685393 16393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:07.686165 16393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:07.687434 16393 hybrid_clock.cc:648] HybridClock initialized: now 1786518667687334 us; error 113 us; skew 500 ppm
Aug 12 07:11:07 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518667, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:07.691099 16393 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:11:07.692212 16393 webserver.cc:460] Webserver started at http://127.31.156.67:40987/ using document root <none> and password file <none>
I20260812 07:11:07.692924 16393 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:07.693002 16393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:07.693264 16393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:07.696069 16393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "9d0b849885f54ee4803723f30e57028d"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "88d90f39ddba88efc8bd734904dd736d"
server_key_iv: "1a2445d74b407b64a4f3d655af44b808"
server_key_version: "encryptionkey@0"
I20260812 07:11:07.696887 16393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "9d0b849885f54ee4803723f30e57028d"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "88d90f39ddba88efc8bd734904dd736d"
server_key_iv: "1a2445d74b407b64a4f3d655af44b808"
server_key_version: "encryptionkey@0"
I20260812 07:11:07.702951 16393 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 07:11:07.706239 16408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:07.708479 16393 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:11:07.708702 16393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "9d0b849885f54ee4803723f30e57028d"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "88d90f39ddba88efc8bd734904dd736d"
server_key_iv: "1a2445d74b407b64a4f3d655af44b808"
server_key_version: "encryptionkey@0"
I20260812 07:11:07.708917 16393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:07.725049 16393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:07.729532 16393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:07.729784 16393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:07.730513 16393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:07.731613 16393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:07.731699 16393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:07.731786 16393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:07.731848 16393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:07.743700 16393 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:33033
I20260812 07:11:07.743848 16541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:33033 every 8 connection(s)
I20260812 07:11:07.745064 16393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:11:07.751689 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 16393
I20260812 07:11:07.751853 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:11:07.752194 32369 external_mini_cluster.cc:1428] Setting key a2f32513f790a2c5e29759632ef75947
Aug 12 07:11:07 dist-test-slave-csg5 krb5kdc[15979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518667, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:07.762959 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.747522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:54639 (local address 127.31.156.126:46033)
0812 07:11:07.747697 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:07.747702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:07.748557 (+ 855us) server_negotiation.cc:407] Connection header received
0812 07:11:07.750312 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:07.750316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:07.750417 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:07.750532 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:07.752426 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.753087 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.754830 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.755059 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.755147 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.758181 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:07.758185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:07.758189 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:07.758226 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:07.760290 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:07.761032 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:07.761036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:07.761037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:07.761150 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:07.761637 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:07.761640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:07.761642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:07.761885 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:07.761991 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:07.762580 (+ 589us) server_negotiation.cc:299] Negotiation successful
0812 07:11:07.762813 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:11:07.764024 16544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.747839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:54639)
0812 07:11:07.748371 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:07.748423 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:07.749968 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:07.750816 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:07.750825 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:07.751269 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:07.752229 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.752256 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.753265 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.753271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.754575 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.754589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.755221 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.755226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.755430 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.756188 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:07.756227 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:07.757977 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:07.760476 (+ 2499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:07.760483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:07.760504 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:07.760900 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:07.761299 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:07.761302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:07.761304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:07.761494 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:07.762007 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:07.762013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:07.762295 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 07:11:07.762500 (+ 205us) client_negotiation.cc:240] Negotiation successful
0812 07:11:07.762938 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":138,"threads_started":1}
I20260812 07:11:07.765795 16542 heartbeater.cc:347] Connected to a master server at 127.31.156.126:46033
I20260812 07:11:07.766093 16542 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:07.766856 16542 heartbeater.cc:511] Master 127.31.156.126:46033 requested a full tablet report, sending...
I20260812 07:11:07.768034 16031 ts_manager.cc:194] Registered new tserver with Master: 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033)
I20260812 07:11:07.768630 16031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:54639
I20260812 07:11:07.779384 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:11:07.791231 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.781898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58276 (local address 127.31.156.126:46033)
0812 07:11:07.782137 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:07.782143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:07.782379 (+ 236us) server_negotiation.cc:407] Connection header received
0812 07:11:07.782577 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:07.782580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:07.782638 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:07.782767 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:07.783602 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.784173 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.785089 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.785331 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.785443 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.787328 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:07.787332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:07.787334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:07.787363 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:07.788995 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:07.789548 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:07.789553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:07.789555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:07.789652 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:07.789988 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:07.789994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:07.789996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:07.790273 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:07.790540 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:07.790890 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 07:11:07.791038 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260812 07:11:07.795683 16031 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:11:07.799404 16031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:11:07.818094 16554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.810520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.31.156.65:34273)
0812 07:11:07.811642 (+ 1122us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:07.811652 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:07.811686 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:11:07.812226 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:07.812231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:07.812439 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:07.812609 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:07.813208 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.814149 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.816034 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.817015 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.817167 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.817373 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:07.817778 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 07:11:07.817923 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004,"thread_start_us":169,"threads_started":1}
I20260812 07:11:07.818145 16556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.810549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:33033 (local address 127.0.0.1:34488)
0812 07:11:07.811801 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:07.811816 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:07.811959 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:07.812795 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:07.812799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:07.812830 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:07.813015 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.813023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.814727 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.814731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.816506 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.816516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.817656 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.817660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.817802 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.817824 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 07:11:07.817909 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 07:11:07.817986 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1174,"mutex_wait_us":85,"spinlock_wait_cycles":23552,"thread_start_us":136,"threads_started":1}
I20260812 07:11:07.818763 16553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.810320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34273 (local address 127.0.0.1:53526)
0812 07:11:07.811121 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:07.811186 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:07.811351 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:07.812795 (+ 1444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:07.812799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:07.812830 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:07.813025 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.813031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.814341 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.814344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.815877 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.815889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.817307 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.817313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.817461 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.817569 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 07:11:07.817698 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 07:11:07.817872 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":73,"spinlock_wait_cycles":4864,"thread_start_us":133,"threads_started":1}
I20260812 07:11:07.818153 16555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.811010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:41829 (local address 127.0.0.1:38394)
0812 07:11:07.811834 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:07.811851 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:07.811959 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:07.812602 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:07.812609 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:07.812634 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:07.812818 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.812826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.814547 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.814551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.816112 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.816124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.817469 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.817477 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.817646 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.817880 (+ 234us) client_negotiation.cc:769] Sending connection context
0812 07:11:07.817956 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 07:11:07.818028 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":179,"spinlock_wait_cycles":4224,"thread_start_us":127,"threads_started":1}
I20260812 07:11:07.819154 16557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.811332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.31.156.67:33033)
0812 07:11:07.811819 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:07.811843 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:07.812108 (+ 265us) server_negotiation.cc:407] Connection header received
0812 07:11:07.812209 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:07.812215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:07.812439 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:07.812619 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:07.813147 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.814562 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.816647 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.817504 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.817638 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.817812 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:07.818557 (+ 745us) server_negotiation.cc:299] Negotiation successful
0812 07:11:07.818739 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":131,"threads_started":1}
I20260812 07:11:07.818830 16558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.811350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38394 (local address 127.31.156.66:41829)
0812 07:11:07.812067 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:07.812082 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:07.812108 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:11:07.812194 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:07.812200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:07.812425 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:07.812609 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:07.812947 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.814377 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.816280 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.817189 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.817319 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.817456 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:07.818494 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0812 07:11:07.818633 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":117,"threads_started":1}
I20260812 07:11:07.822192 16314 tablet_service.cc:1467] Processing CreateTablet for tablet eda86befac3f4409811ba30e9f22b040 (DEFAULT_TABLE table=test-table [id=e68e949b2bfb4ea6b5f74232b1ae0ae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:07.822484 16462 tablet_service.cc:1467] Processing CreateTablet for tablet eda86befac3f4409811ba30e9f22b040 (DEFAULT_TABLE table=test-table [id=e68e949b2bfb4ea6b5f74232b1ae0ae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:07.822479 16171 tablet_service.cc:1467] Processing CreateTablet for tablet eda86befac3f4409811ba30e9f22b040 (DEFAULT_TABLE table=test-table [id=e68e949b2bfb4ea6b5f74232b1ae0ae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:07.823840 16462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda86befac3f4409811ba30e9f22b040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:07.823832 16171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda86befac3f4409811ba30e9f22b040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:07.824261 16314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda86befac3f4409811ba30e9f22b040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:07.832434 16559 tablet_bootstrap.cc:492] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5: Bootstrap starting.
I20260812 07:11:07.833942 16561 tablet_bootstrap.cc:492] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d: Bootstrap starting.
I20260812 07:11:07.835024 16560 tablet_bootstrap.cc:492] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Bootstrap starting.
I20260812 07:11:07.835954 16559 tablet_bootstrap.cc:654] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:07.836829 16559 log.cc:824] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:07.836936 16561 tablet_bootstrap.cc:654] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:07.837523 16560 tablet_bootstrap.cc:654] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:07.838486 16560 log.cc:824] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:07.838486 16561 log.cc:824] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:07.839299 16559 tablet_bootstrap.cc:492] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5: No bootstrap required, opened a new log
I20260812 07:11:07.839548 16559 ts_tablet_manager.cc:1397] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:11:07.840706 16560 tablet_bootstrap.cc:492] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: No bootstrap required, opened a new log
I20260812 07:11:07.840885 16561 tablet_bootstrap.cc:492] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d: No bootstrap required, opened a new log
I20260812 07:11:07.840962 16560 ts_tablet_manager.cc:1397] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 07:11:07.841095 16561 ts_tablet_manager.cc:1397] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:11:07.844254 16559 raft_consensus.cc:348] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } }
I20260812 07:11:07.844623 16559 raft_consensus.cc:374] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:07.844707 16559 raft_consensus.cc:729] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28668e338ddd4aacaacba83a4f86fbb5, State: Initialized, Role: FOLLOWER
I20260812 07:11:07.845286 16559 consensus_queue.cc:260] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } }
I20260812 07:11:07.845424 16560 raft_consensus.cc:348] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } }
I20260812 07:11:07.845782 16560 raft_consensus.cc:374] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:07.845880 16560 raft_consensus.cc:729] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf44c8f45e49ac8d4eb7129cfdcca2, State: Initialized, Role: FOLLOWER
I20260812 07:11:07.845902 16561 raft_consensus.cc:348] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } }
I20260812 07:11:07.846342 16561 raft_consensus.cc:374] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:07.846429 16561 raft_consensus.cc:729] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d0b849885f54ee4803723f30e57028d, State: Initialized, Role: FOLLOWER
I20260812 07:11:07.846400 16560 consensus_queue.cc:260] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } }
I20260812 07:11:07.846761 16559 ts_tablet_manager.cc:1428] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 07:11:07.847180 16388 heartbeater.cc:503] Master 127.31.156.126:46033 was elected leader, sending a full tablet report...
I20260812 07:11:07.847227 16561 consensus_queue.cc:260] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } }
I20260812 07:11:07.847683 16560 ts_tablet_manager.cc:1428] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 07:11:07.848100 16542 heartbeater.cc:503] Master 127.31.156.126:46033 was elected leader, sending a full tablet report...
I20260812 07:11:07.848356 16561 ts_tablet_manager.cc:1428] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 07:11:07.848450 16239 heartbeater.cc:503] Master 127.31.156.126:46033 was elected leader, sending a full tablet report...
I20260812 07:11:07.855883 16565 raft_consensus.cc:482] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:11:07.856109 16565 raft_consensus.cc:504] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } }
I20260812 07:11:07.857739 16565 leader_election.cc:288] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89bf44c8f45e49ac8d4eb7129cfdcca2 (127.31.156.65:34273), 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033)
I20260812 07:11:07.863802 16557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.858699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:40031 (local address 127.31.156.67:33033)
0812 07:11:07.858835 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:07.858839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:07.858852 (+ 13us) server_negotiation.cc:407] Connection header received
0812 07:11:07.858909 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:07.858911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:07.858949 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:07.859042 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:07.859388 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.860280 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.862357 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.863086 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.863219 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.863272 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:07.863561 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 07:11:07.863658 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 07:11:07.863847 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.858007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:33033 (local address 127.31.156.66:40031)
0812 07:11:07.858147 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:07.858186 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:07.858289 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:07.859056 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:07.859060 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:07.859077 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:07.859248 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.859255 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.860430 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.860436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.862216 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.862226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.863225 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.863229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.863382 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.863411 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:11:07.863522 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 07:11:07.863651 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":235}
I20260812 07:11:07.863866 16554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.858090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:41297 (local address 127.31.156.65:34273)
0812 07:11:07.858217 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:07.858222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:07.858609 (+ 387us) server_negotiation.cc:407] Connection header received
0812 07:11:07.859216 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:07.859220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:07.859385 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:07.859485 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:07.859746 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.860254 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.862328 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.863050 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.863197 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.863258 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:07.863652 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 07:11:07.863746 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:11:07.864423 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.858026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34273 (local address 127.31.156.66:41297)
0812 07:11:07.858507 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:07.858519 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:07.858657 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:07.859494 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:07.859497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:07.859513 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:07.859628 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.859641 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.860519 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.860523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.862046 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:07.862058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.863360 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.863364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:07.863499 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.863529 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:11:07.864084 (+ 555us) client_negotiation.cc:240] Negotiation successful
0812 07:11:07.864211 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":169,"thread_start_us":159,"threads_started":1}
I20260812 07:11:07.864849 16483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda86befac3f4409811ba30e9f22b040" candidate_uuid: "28668e338ddd4aacaacba83a4f86fbb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d0b849885f54ee4803723f30e57028d" is_pre_election: true
I20260812 07:11:07.865234 16483 raft_consensus.cc:2454] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28668e338ddd4aacaacba83a4f86fbb5 in term 0.
I20260812 07:11:07.865897 16267 leader_election.cc:302] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28668e338ddd4aacaacba83a4f86fbb5, 9d0b849885f54ee4803723f30e57028d; no voters:
I20260812 07:11:07.866127 16192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda86befac3f4409811ba30e9f22b040" candidate_uuid: "28668e338ddd4aacaacba83a4f86fbb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" is_pre_election: true
I20260812 07:11:07.866351 16565 raft_consensus.cc:2781] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:11:07.866433 16192 raft_consensus.cc:2454] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28668e338ddd4aacaacba83a4f86fbb5 in term 0.
I20260812 07:11:07.866472 16565 raft_consensus.cc:482] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:11:07.866550 16565 raft_consensus.cc:3037] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:07.867853 16565 raft_consensus.cc:504] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } }
I20260812 07:11:07.868310 16565 leader_election.cc:288] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [CANDIDATE]: Term 1 election: Requested vote from peers 89bf44c8f45e49ac8d4eb7129cfdcca2 (127.31.156.65:34273), 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033)
I20260812 07:11:07.868721 16192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda86befac3f4409811ba30e9f22b040" candidate_uuid: "28668e338ddd4aacaacba83a4f86fbb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2"
I20260812 07:11:07.868793 16483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda86befac3f4409811ba30e9f22b040" candidate_uuid: "28668e338ddd4aacaacba83a4f86fbb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d0b849885f54ee4803723f30e57028d"
I20260812 07:11:07.868883 16192 raft_consensus.cc:3037] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:07.868949 16483 raft_consensus.cc:3037] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:07.870128 16192 raft_consensus.cc:2454] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28668e338ddd4aacaacba83a4f86fbb5 in term 1.
I20260812 07:11:07.870270 16483 raft_consensus.cc:2454] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28668e338ddd4aacaacba83a4f86fbb5 in term 1.
I20260812 07:11:07.870575 16263 leader_election.cc:302] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28668e338ddd4aacaacba83a4f86fbb5, 89bf44c8f45e49ac8d4eb7129cfdcca2; no voters:
I20260812 07:11:07.870823 16565 raft_consensus.cc:2781] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:11:07.871107 16565 raft_consensus.cc:686] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 1 LEADER]: Becoming Leader. State: Replica: 28668e338ddd4aacaacba83a4f86fbb5, State: Running, Role: LEADER
I20260812 07:11:07.871533 16565 consensus_queue.cc:237] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } }
I20260812 07:11:07.875432 16030 catalog_manager.cc:5495] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66). New cstate: current_term: 1 leader_uuid: "28668e338ddd4aacaacba83a4f86fbb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260812 07:11:07.902915 16558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.898428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.31.156.66:41829)
0812 07:11:07.898670 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:07.898700 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:07.898724 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:11:07.898835 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:07.898838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:07.898912 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:07.899018 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:11:07.899294 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.899862 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.900788 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:07.901069 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:07.901174 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:07.901631 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:11:07.902208 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:11:07.902330 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:07.902570 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 07:11:07.902646 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260812 07:11:07.910178 16483 raft_consensus.cc:1261] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d [term 1 FOLLOWER]: Refusing update from remote peer 28668e338ddd4aacaacba83a4f86fbb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:07.910178 16192 raft_consensus.cc:1261] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Refusing update from remote peer 28668e338ddd4aacaacba83a4f86fbb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:07.911253 16569 consensus_queue.cc:1035] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:07.911619 16565 consensus_queue.cc:1035] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:07.921729 16574 mvcc.cc:205] Tried to move back new op lower bound from 7317580463750987776 to 7317580463604748288. Current Snapshot: MvccSnapshot[applied={T|T < 7317580463750987776}]
I20260812 07:11:07.922320 16575 mvcc.cc:205] Tried to move back new op lower bound from 7317580463750987776 to 7317580463604748288. Current Snapshot: MvccSnapshot[applied={T|T < 7317580463750987776}]
I20260812 07:11:07.932523 16028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:11:07.932731 16028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:11:07.936420 16028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Sending DeleteTablet for 3 replicas of tablet eda86befac3f4409811ba30e9f22b040
I20260812 07:11:07.937688 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 16097
I20260812 07:11:07.937678 16462 tablet_service.cc:1514] Processing DeleteTablet for tablet eda86befac3f4409811ba30e9f22b040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:07 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:34488
I20260812 07:11:07.937650 16314 tablet_service.cc:1514] Processing DeleteTablet for tablet eda86befac3f4409811ba30e9f22b040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:07 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:38394
I20260812 07:11:07.937711 16171 tablet_service.cc:1514] Processing DeleteTablet for tablet eda86befac3f4409811ba30e9f22b040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:07 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:53526
I20260812 07:11:07.942059 16584 tablet_replica.cc:330] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d: stopping tablet replica
I20260812 07:11:07.942747 16584 raft_consensus.cc:2229] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:11:07.943300 16584 raft_consensus.cc:2258] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:07.943878 16583 tablet_replica.cc:330] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5: stopping tablet replica
I20260812 07:11:07.944553 16583 raft_consensus.cc:2229] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:11:07.945494 16583 raft_consensus.cc:2258] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:07.948868 16584 ts_tablet_manager.cc:1905] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:11:07.952164 16583 ts_tablet_manager.cc:1905] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:11:07.952895 16584 ts_tablet_manager.cc:1918] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:07.953022 16584 log.cc:1196] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/eda86befac3f4409811ba30e9f22b040
I20260812 07:11:07.953415 16584 ts_tablet_manager.cc:1939] T eda86befac3f4409811ba30e9f22b040 P 9d0b849885f54ee4803723f30e57028d: Deleting consensus metadata
I20260812 07:11:07.955106 16014 catalog_manager.cc:4841] TS 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033): tablet eda86befac3f4409811ba30e9f22b040 (table test-table [id=e68e949b2bfb4ea6b5f74232b1ae0ae8]) successfully deleted
I20260812 07:11:07.955325 16583 ts_tablet_manager.cc:1918] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:07.955447 16583 log.cc:1196] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/eda86befac3f4409811ba30e9f22b040
I20260812 07:11:07.955781 16583 ts_tablet_manager.cc:1939] T eda86befac3f4409811ba30e9f22b040 P 28668e338ddd4aacaacba83a4f86fbb5: Deleting consensus metadata
I20260812 07:11:07.957006 16013 catalog_manager.cc:4841] TS 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829): tablet eda86befac3f4409811ba30e9f22b040 (table test-table [id=e68e949b2bfb4ea6b5f74232b1ae0ae8]) successfully deleted
W20260812 07:11:07.959821 16011 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:34273 (error 108)
I20260812 07:11:07.960248 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 16244
I20260812 07:11:07.965188 16555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:07.964678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34273 (local address 127.0.0.1:53536)
0812 07:11:07.964904 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:07.965024 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260812 07:11:07.965505 16011 catalog_manager.cc:4584] TS 89bf44c8f45e49ac8d4eb7129cfdcca2 (127.31.156.65:34273): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eda86befac3f4409811ba30e9f22b040: Network error: Client connection negotiation failed: client connection to 127.31.156.65:34273: connect: Connection refused (error 111)
I20260812 07:11:07.971550 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 16393
I20260812 07:11:07.985536 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 15993
I20260812 07:11:07.998589 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46033
--webserver_interface=127.31.156.126
--webserver_port=34173
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:46033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:08.127763 16587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:08.128136 16587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:08.128187 16587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:08.132853 16587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:11:08.132948 16587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:08.132969 16587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:08.132987 16587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:11:08.133003 16587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:11:08.139169 16587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:46033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=34173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:08.140484 16587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:08.141659 16587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:08.158932 16594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:08.159176 16587 server_base.cc:1034] running on GCE node
W20260812 07:11:08.158978 16595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:08.158946 16599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:08.160017 16587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:08.161402 16587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:08.162645 16587 hybrid_clock.cc:648] HybridClock initialized: now 1786518668162626 us; error 39 us; skew 500 ppm
Aug 12 07:11:08 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518668, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:08.166553 16587 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:11:08.167886 16587 webserver.cc:460] Webserver started at http://127.31.156.126:34173/ using document root <none> and password file <none>
I20260812 07:11:08.168645 16587 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:08.168800 16587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:08.173960 16587 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:11:08.176965 16607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:08.178943 16587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:08.179148 16587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "ad7107e2ce0e4c749ebf6ab04768b72f"
format_stamp: "Formatted at 2026-08-12 07:11:06 on dist-test-slave-csg5"
server_key: "03a8481afce765a266bd1eb896a37093"
server_key_iv: "0a9b4920c4388dc5ed405bf77fd9ae41"
server_key_version: "encryptionkey@0"
I20260812 07:11:08.180038 16587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:08.198740 16587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:08.203478 16587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:08.203763 16587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:08.213474 16587 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:46033
I20260812 07:11:08.213857 16665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:46033 every 8 connection(s)
I20260812 07:11:08.215003 16587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:11:08.219740 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 16587
I20260812 07:11:08.220286 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:34273
--local_ip_for_outbound_sockets=127.31.156.65
--tserver_master_addrs=127.31.156.126:46033
--webserver_port=35421
--webserver_interface=127.31.156.65
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:11:08.220448 16666 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:11:08.222820 16666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Bootstrap starting.
I20260812 07:11:08.252399 16666 log.cc:824] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:08.265026 16666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:11:08.265596 16666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Bootstrap complete.
I20260812 07:11:08.271142 16666 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:08.271716 16666 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad7107e2ce0e4c749ebf6ab04768b72f, State: Initialized, Role: FOLLOWER
I20260812 07:11:08.272385 16666 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:08.272567 16666 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:11:08.272681 16666 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:11:08.272833 16666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:11:08.275468 16666 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:08.275918 16666 leader_election.cc:302] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad7107e2ce0e4c749ebf6ab04768b72f; no voters:
I20260812 07:11:08.276321 16666 leader_election.cc:288] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:11:08.276510 16670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:11:08.276789 16670 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 2 LEADER]: Becoming Leader. State: Replica: ad7107e2ce0e4c749ebf6ab04768b72f, State: Running, Role: LEADER
I20260812 07:11:08.277148 16670 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:08.277864 16666 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:11:08.278795 16671 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } } }
I20260812 07:11:08.278956 16671 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:08.279261 16672 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad7107e2ce0e4c749ebf6ab04768b72f. Latest consensus state: current_term: 2 leader_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } } }
I20260812 07:11:08.279853 16672 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:08.280174 16679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:11:08.285709 16679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:11:08.286482 16679 catalog_manager.cc:1259] Loaded cluster ID: a390eadcc7f5481ebd5149f3bdc2b774
I20260812 07:11:08.286557 16679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:11:08.288493 16679 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 07:11:08.293018 16679 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 07:11:08.299899 16679 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:11:08.300940 16679 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Loaded TSK: 0
I20260812 07:11:08.302394 16679 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:11:08.379081 16668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:08.379601 16668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:08.379792 16668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:08.384456 16668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:08.384583 16668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:08.384938 16668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:11:08.392201 16668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:34273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=35421
--tserver_master_addrs=127.31.156.126:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:08.393537 16668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:08.394910 16668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:08.407546 16695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:08.407524 16696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:08.407922 16668 server_base.cc:1034] running on GCE node
W20260812 07:11:08.407522 16698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:08.408656 16668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:08.409535 16668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:08.410753 16668 hybrid_clock.cc:648] HybridClock initialized: now 1786518668410729 us; error 47 us; skew 500 ppm
Aug 12 07:11:08 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518668, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:08.414072 16668 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:11:08.415258 16668 webserver.cc:460] Webserver started at http://127.31.156.65:35421/ using document root <none> and password file <none>
I20260812 07:11:08.415921 16668 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:08.416009 16668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:08.420552 16668 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:11:08.423297 16705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:08.424983 16668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:08.425283 16668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "282de14ccab1062e50213a4bdd19c17b"
server_key_iv: "ac4a9d4b13cc76b7ee3d0b3ac8931354"
server_key_version: "encryptionkey@0"
I20260812 07:11:08.425962 16668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:08.447170 16668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:08.450999 16668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:08.451231 16668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:08.452024 16668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:08.453181 16712 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 07:11:08.458158 16668 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 07:11:08.458254 16668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 07:11:08.458315 16668 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 07:11:08.460342 16668 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 07:11:08.460403 16668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260812 07:11:08.460748 16712 tablet_bootstrap.cc:492] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Bootstrap starting.
I20260812 07:11:08.472510 16668 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:34273
I20260812 07:11:08.472630 16831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:34273 every 8 connection(s)
I20260812 07:11:08.473676 16668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:11:08.482151 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 16668
I20260812 07:11:08.482640 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:41829
--local_ip_for_outbound_sockets=127.31.156.66
--tserver_master_addrs=127.31.156.126:46033
--webserver_port=36979
--webserver_interface=127.31.156.66
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:11:08.484620 16712 log.cc:824] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:11:08 dist-test-slave-csg5 krb5kdc[15979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518668, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:08.495072 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:08.476094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:36699 (local address 127.31.156.126:46033)
0812 07:11:08.476510 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:08.476551 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:08.477000 (+ 449us) server_negotiation.cc:407] Connection header received
0812 07:11:08.478373 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:08.478389 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:08.478830 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:08.479252 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:08.480735 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.481986 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.484080 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.484973 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.485213 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:08.489173 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:08.489179 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:08.489195 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:08.489233 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:08.491433 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:08.492092 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:08.492098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:08.492102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:08.492238 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:08.492644 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:08.492648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:08.492649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:08.493045 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:08.493205 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:08.493782 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 07:11:08.494065 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":143,"threads_started":1}
I20260812 07:11:08.495088 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:08.476161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:36699)
0812 07:11:08.476799 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:08.476858 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:08.477792 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:08.479317 (+ 1525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:08.479326 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:08.479998 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:08.480521 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:08.480539 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.482218 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.482222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:08.483887 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:08.483906 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.485193 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.485199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:08.485746 (+ 547us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:08.486741 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:08.486763 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:08.488862 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:08.491605 (+ 2743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:08.491611 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:08.491625 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:08.491933 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:08.492349 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:08.492352 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:08.492354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:08.492521 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:08.493174 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:08.493181 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:08.493451 (+ 270us) client_negotiation.cc:769] Sending connection context
0812 07:11:08.493648 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 07:11:08.494065 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":212,"threads_started":1}
I20260812 07:11:08.495988 16712 tablet_bootstrap.cc:492] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:11:08.496594 16712 tablet_bootstrap.cc:492] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Bootstrap complete.
I20260812 07:11:08.498430 16832 heartbeater.cc:347] Connected to a master server at 127.31.156.126:46033
I20260812 07:11:08.498446 16712 ts_tablet_manager.cc:1397] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Time spent bootstrapping tablet: real 0.038s user 0.032s sys 0.003s
I20260812 07:11:08.498796 16832 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:08.499488 16832 heartbeater.cc:511] Master 127.31.156.126:46033 requested a full tablet report, sending...
I20260812 07:11:08.501839 16712 raft_consensus.cc:348] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } }
I20260812 07:11:08.502476 16712 raft_consensus.cc:729] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf44c8f45e49ac8d4eb7129cfdcca2, State: Initialized, Role: FOLLOWER
I20260812 07:11:08.502424 16626 ts_manager.cc:194] Registered new tserver with Master: 89bf44c8f45e49ac8d4eb7129cfdcca2 (127.31.156.65:34273)
I20260812 07:11:08.503073 16712 consensus_queue.cc:260] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } }
I20260812 07:11:08.504213 16712 ts_tablet_manager.cc:1428] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 07:11:08.506302 16626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:36699
I20260812 07:11:08.509516 16832 heartbeater.cc:503] Master 127.31.156.126:46033 was elected leader, sending a full tablet report...
Aug 12 07:11:08 dist-test-slave-csg5 krb5kdc[15979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518668, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:11:08.518316 16839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:08.505833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34273 (local address 127.0.0.1:53546)
0812 07:11:08.506689 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:08.506742 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:08.506895 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:08.507507 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:08.507514 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:08.507903 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:08.508017 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:08.508024 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.509549 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.509553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:08.510597 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:08.510612 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.511726 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.511730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:08.511876 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:08.512517 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:08.512531 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:08.514046 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:08.516232 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:08.516252 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:08.516258 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:08.516534 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:08.516891 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:08.516894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:08.516896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:08.516996 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:08.517524 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:08.517530 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:08.517672 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 07:11:08.517831 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 07:11:08.518078 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":145,"threads_started":1}
I20260812 07:11:08.518316 16841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:08.505924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53546 (local address 127.31.156.65:34273)
0812 07:11:08.506383 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:08.506398 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:08.506857 (+ 459us) server_negotiation.cc:407] Connection header received
0812 07:11:08.507055 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:08.507061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:08.507279 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:08.507485 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:08.508150 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.509381 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.510816 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.511558 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.511703 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:08.514224 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:08.514239 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:08.514254 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:08.514279 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:08.516087 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:08.516679 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:08.516685 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:08.516686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:08.516784 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:08.517107 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:08.517111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:08.517112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:08.517397 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:08.517548 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:08.517921 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 07:11:08.518106 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":124,"threads_started":1}
I20260812 07:11:08.519578 16761 tablet_service.cc:1514] Processing DeleteTablet for tablet eda86befac3f4409811ba30e9f22b040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:07 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:53546
I20260812 07:11:08.520520 16842 tablet_replica.cc:330] stopping tablet replica
I20260812 07:11:08.520859 16842 raft_consensus.cc:2229] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:11:08.521134 16842 raft_consensus.cc:2258] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:08.522145 16842 ts_tablet_manager.cc:1905] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:11:08.526135 16842 ts_tablet_manager.cc:1918] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:08.526250 16842 log.cc:1196] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal/wals/eda86befac3f4409811ba30e9f22b040
I20260812 07:11:08.526580 16842 ts_tablet_manager.cc:1939] T eda86befac3f4409811ba30e9f22b040 P 89bf44c8f45e49ac8d4eb7129cfdcca2: Deleting consensus metadata
I20260812 07:11:08.528546 16609 catalog_manager.cc:4841] TS 89bf44c8f45e49ac8d4eb7129cfdcca2 (127.31.156.65:34273): tablet eda86befac3f4409811ba30e9f22b040 (table test-table [id=e68e949b2bfb4ea6b5f74232b1ae0ae8]) successfully deleted
W20260812 07:11:08.623567 16836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:08.623939 16836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:08.624027 16836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:08.628935 16836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:08.629073 16836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:08.629186 16836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:11:08.635952 16836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:41829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=36979
--tserver_master_addrs=127.31.156.126:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:08.637339 16836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:08.638571 16836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:08.650769 16847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:08.650816 16850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:08.650748 16848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:08.651085 16836 server_base.cc:1034] running on GCE node
I20260812 07:11:08.651569 16836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:08.652331 16836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:08.653539 16836 hybrid_clock.cc:648] HybridClock initialized: now 1786518668653475 us; error 79 us; skew 500 ppm
Aug 12 07:11:08 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518668, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:08.657403 16836 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:11:08.658618 16836 webserver.cc:460] Webserver started at http://127.31.156.66:36979/ using document root <none> and password file <none>
I20260812 07:11:08.659408 16836 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:08.659535 16836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:08.664594 16836 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:11:08.667302 16858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:08.668964 16836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:08.669162 16836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "28668e338ddd4aacaacba83a4f86fbb5"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "22597b2c07b93c092763500ce89a25bc"
server_key_iv: "771ee5778821402c8e76df107f505e55"
server_key_version: "encryptionkey@0"
I20260812 07:11:08.669867 16836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:08.686012 16836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:08.691522 16836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:08.691812 16836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:08.692586 16836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:08.693661 16836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:08.693753 16836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:08.693842 16836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:08.693900 16836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:08.706094 16836 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:41829
I20260812 07:11:08.706108 16979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:41829 every 8 connection(s)
I20260812 07:11:08.707440 16836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:11:08.715538 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 16836
I20260812 07:11:08.716172 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:33033
--local_ip_for_outbound_sockets=127.31.156.67
--tserver_master_addrs=127.31.156.126:46033
--webserver_port=40987
--webserver_interface=127.31.156.67
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:11:08 dist-test-slave-csg5 krb5kdc[15979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518668, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:08.726593 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:08.709896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:35685 (local address 127.31.156.126:46033)
0812 07:11:08.710071 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:08.710075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:08.711312 (+ 1237us) server_negotiation.cc:407] Connection header received
0812 07:11:08.712968 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:08.712972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:08.713051 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:08.713186 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:08.714735 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.715341 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.717005 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.717546 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.717719 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:08.721006 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:08.721013 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:08.721020 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:08.721063 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:08.723304 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:08.724140 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:08.724148 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:08.724152 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:08.724269 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:08.724941 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:08.724947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:08.724952 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:08.725296 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:08.725440 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:08.726120 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 07:11:08.726314 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:11:08.727540 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:08.710266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:35685)
0812 07:11:08.711001 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:08.711070 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:08.712682 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:08.713422 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:08.713430 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:08.713910 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:08.714495 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:08.714511 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.715522 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.715525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:08.716840 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:08.716854 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.717779 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.717783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:08.718037 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:08.718866 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:08.718885 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:08.720739 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:08.723578 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:08.723586 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:08.723605 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:08.723999 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:08.724482 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:08.724485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:08.724486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:08.724806 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:08.725489 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:08.725496 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:08.725812 (+ 316us) client_negotiation.cc:769] Sending connection context
0812 07:11:08.726022 (+ 210us) client_negotiation.cc:240] Negotiation successful
0812 07:11:08.726458 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":135,"threads_started":1}
I20260812 07:11:08.729110 16981 heartbeater.cc:347] Connected to a master server at 127.31.156.126:46033
I20260812 07:11:08.729466 16981 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:08.730290 16981 heartbeater.cc:511] Master 127.31.156.126:46033 requested a full tablet report, sending...
I20260812 07:11:08.731895 16626 ts_manager.cc:194] Registered new tserver with Master: 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829)
I20260812 07:11:08.732903 16626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:35685
W20260812 07:11:08.854887 16985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:08.855257 16985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:08.855345 16985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:08.859807 16985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:08.859915 16985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:08.860015 16985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:11:08.865844 16985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:33033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=40987
--tserver_master_addrs=127.31.156.126:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:08.867197 16985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:08.868389 16985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:08.880003 16990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:08.880020 16992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:08.879995 16996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:08.880630 16985 server_base.cc:1034] running on GCE node
I20260812 07:11:08.881182 16985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:08.881894 16985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:08.883076 16985 hybrid_clock.cc:648] HybridClock initialized: now 1786518668883058 us; error 34 us; skew 500 ppm
Aug 12 07:11:08 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518668, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:08.886722 16985 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:11:08.887814 16985 webserver.cc:460] Webserver started at http://127.31.156.67:40987/ using document root <none> and password file <none>
I20260812 07:11:08.888475 16985 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:08.888585 16985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:08.893692 16985 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 07:11:08.896947 17003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:08.899048 16985 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 07:11:08.899251 16985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "9d0b849885f54ee4803723f30e57028d"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "88d90f39ddba88efc8bd734904dd736d"
server_key_iv: "1a2445d74b407b64a4f3d655af44b808"
server_key_version: "encryptionkey@0"
I20260812 07:11:08.899932 16985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:08.927668 16985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:08.932549 16985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:08.932842 16985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:08.933591 16985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:08.934777 16985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:08.934850 16985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:08.934933 16985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:08.934978 16985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:08.946942 16985 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:33033
I20260812 07:11:08.946961 17128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:33033 every 8 connection(s)
I20260812 07:11:08.948208 16985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:11:08.958271 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 16985
Aug 12 07:11:08 dist-test-slave-csg5 krb5kdc[15979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518668, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:08.967334 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:08.950388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:54713 (local address 127.31.156.126:46033)
0812 07:11:08.950554 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:08.950558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:08.951747 (+ 1189us) server_negotiation.cc:407] Connection header received
0812 07:11:08.953553 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:08.953557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:08.953631 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:08.953760 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:08.955396 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.955995 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.957589 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.957884 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.958023 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:08.961776 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:08.961780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:08.961783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:08.961817 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:08.963806 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:08.964752 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:08.964756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:08.964759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:08.964881 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:08.965339 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:08.965343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:08.965345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:08.965626 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:08.965708 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:08.966967 (+ 1259us) server_negotiation.cc:299] Negotiation successful
0812 07:11:08.967185 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65,"spinlock_wait_cycles":40704}
I20260812 07:11:08.968644 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:08.950925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:54713)
0812 07:11:08.951524 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:08.951579 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:08.953271 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:08.954001 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:08.954009 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:08.954500 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:08.955167 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:08.955185 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.956174 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.956178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:08.957420 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:08.957433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.958111 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.958115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:08.958329 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:08.959496 (+ 1167us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:08.959517 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:08.961518 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:08.963975 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:08.963982 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:08.964003 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:08.964610 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:08.965001 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:08.965004 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:08.965005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:08.965205 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:08.965885 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:08.965897 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:08.966475 (+ 578us) client_negotiation.cc:769] Sending connection context
0812 07:11:08.966825 (+ 350us) client_negotiation.cc:240] Negotiation successful
0812 07:11:08.967338 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":134,"threads_started":1}
I20260812 07:11:08.970218 17134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:08.959161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.31.156.126:46033)
0812 07:11:08.959489 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:08.959496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:08.959616 (+ 120us) server_negotiation.cc:407] Connection header received
0812 07:11:08.960053 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:08.960056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:08.960115 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:08.960214 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:08.961053 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.961887 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.963263 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.963543 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.963703 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:08.965635 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:08.965639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:08.965641 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:08.965807 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:08.967872 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:08.968361 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:08.968365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:08.968367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:08.968477 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:08.968877 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:08.968882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:08.968884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:08.969229 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:08.969583 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:08.969857 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 07:11:08.970013 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":97,"threads_started":1}
I20260812 07:11:08.970701 17129 heartbeater.cc:347] Connected to a master server at 127.31.156.126:46033
I20260812 07:11:08.971019 17129 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:08.971649 17129 heartbeater.cc:511] Master 127.31.156.126:46033 requested a full tablet report, sending...
I20260812 07:11:08.973088 16626 ts_manager.cc:194] Registered new tserver with Master: 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033)
I20260812 07:11:08.973842 16626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:54713
I20260812 07:11:08.985497 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:11:08.990909 17134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:08.986366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58300 (local address 127.31.156.126:46033)
0812 07:11:08.986516 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:08.986520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:08.986753 (+ 233us) server_negotiation.cc:407] Connection header received
0812 07:11:08.986924 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:08.986926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:08.987046 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:08.987149 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:11:08.987487 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.988103 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.989125 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:08.989395 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:08.989496 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:08.989881 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:11:08.990346 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:11:08.990453 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:08.990626 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 07:11:08.990774 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:11:08.991833 16626 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:11:08.995484 16626 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:11:09.005400 16761 tablet_service.cc:1467] Processing CreateTablet for tablet 49ec41199977468c9e6c850388011ffd (DEFAULT_TABLE table=test-table [id=a55443de81ba4140a81b8f8ee465ee43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:09.006248 16761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49ec41199977468c9e6c850388011ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:09.012141 17143 tablet_bootstrap.cc:492] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2: Bootstrap starting.
I20260812 07:11:09.012588 16839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.003906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:41829 (local address 127.0.0.1:38402)
0812 07:11:09.004150 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:09.004169 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:09.004289 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:09.005350 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:09.005354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:09.005375 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:09.005497 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:09.005503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.007603 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.007607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:09.008802 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:09.008817 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.011623 (+ 2806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.011631 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:09.011779 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:09.011867 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 07:11:09.011972 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 07:11:09.012142 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260812 07:11:09.013309 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.004245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.31.156.66:41829)
0812 07:11:09.004641 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:09.004657 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:09.004767 (+ 110us) server_negotiation.cc:407] Connection header received
0812 07:11:09.004872 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:09.004879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:09.005172 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:09.005369 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:09.005940 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.007415 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.008962 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.011296 (+ 2334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.011477 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:09.011724 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:09.012965 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0812 07:11:09.013140 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":165,"threads_started":1}
I20260812 07:11:09.014101 17143 tablet_bootstrap.cc:654] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:09.014245 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.004185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:33033 (local address 127.0.0.1:34490)
0812 07:11:09.004646 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:09.004659 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:09.004762 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:09.005743 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:09.005746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:09.005765 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:09.005877 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:09.005883 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.007687 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.007697 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:09.008543 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:09.008556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.012021 (+ 3465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.012025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:09.013084 (+ 1059us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:09.013117 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:11:09.013746 (+ 629us) client_negotiation.cc:240] Negotiation successful
0812 07:11:09.013928 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":167,"threads_started":1}
I20260812 07:11:09.014741 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.004721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34490 (local address 127.31.156.67:33033)
0812 07:11:09.005232 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:09.005247 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:09.005273 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:11:09.005352 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:09.005357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:09.005549 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:09.006034 (+ 485us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:09.006385 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.007500 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.008721 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.011586 (+ 2865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.011717 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:09.011868 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:09.013982 (+ 2114us) server_negotiation.cc:299] Negotiation successful
0812 07:11:09.014499 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":129,"threads_started":1}
I20260812 07:11:09.016811 17143 tablet_bootstrap.cc:492] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2: No bootstrap required, opened a new log
I20260812 07:11:09.016999 17143 ts_tablet_manager.cc:1397] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 07:11:09.017558 17143 raft_consensus.cc:348] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:09.017704 17143 raft_consensus.cc:374] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:09.017746 17143 raft_consensus.cc:729] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf44c8f45e49ac8d4eb7129cfdcca2, State: Initialized, Role: FOLLOWER
I20260812 07:11:09.017911 17143 consensus_queue.cc:260] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:09.018373 17143 ts_tablet_manager.cc:1428] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 07:11:09.018260 16910 tablet_service.cc:1467] Processing CreateTablet for tablet 49ec41199977468c9e6c850388011ffd (DEFAULT_TABLE table=test-table [id=a55443de81ba4140a81b8f8ee465ee43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:09.019995 16910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49ec41199977468c9e6c850388011ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:09.019616 17053 tablet_service.cc:1467] Processing CreateTablet for tablet 49ec41199977468c9e6c850388011ffd (DEFAULT_TABLE table=test-table [id=a55443de81ba4140a81b8f8ee465ee43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:09.020690 17053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49ec41199977468c9e6c850388011ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:09.029072 17147 tablet_bootstrap.cc:492] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d: Bootstrap starting.
I20260812 07:11:09.029277 17150 tablet_bootstrap.cc:492] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: Bootstrap starting.
I20260812 07:11:09.032284 17147 tablet_bootstrap.cc:654] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:09.032301 17150 tablet_bootstrap.cc:654] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:09.033188 17150 log.cc:824] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:09.033231 17147 log.cc:824] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:09.036340 17150 tablet_bootstrap.cc:492] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: No bootstrap required, opened a new log
I20260812 07:11:09.036347 17147 tablet_bootstrap.cc:492] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d: No bootstrap required, opened a new log
I20260812 07:11:09.036612 17150 ts_tablet_manager.cc:1397] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:11:09.036612 17147 ts_tablet_manager.cc:1397] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260812 07:11:09.041508 17147 raft_consensus.cc:348] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:09.041508 17150 raft_consensus.cc:348] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:09.041901 17147 raft_consensus.cc:374] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:09.041901 17150 raft_consensus.cc:374] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:09.041980 17150 raft_consensus.cc:729] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28668e338ddd4aacaacba83a4f86fbb5, State: Initialized, Role: FOLLOWER
I20260812 07:11:09.041996 17147 raft_consensus.cc:729] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d0b849885f54ee4803723f30e57028d, State: Initialized, Role: FOLLOWER
I20260812 07:11:09.042703 17147 consensus_queue.cc:260] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:09.042702 17150 consensus_queue.cc:260] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:09.043632 17129 heartbeater.cc:503] Master 127.31.156.126:46033 was elected leader, sending a full tablet report...
I20260812 07:11:09.043970 16981 heartbeater.cc:503] Master 127.31.156.126:46033 was elected leader, sending a full tablet report...
I20260812 07:11:09.044083 17150 ts_tablet_manager.cc:1428] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 07:11:09.044083 17147 ts_tablet_manager.cc:1428] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:11:09.124852 17146 raft_consensus.cc:482] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:11:09.125097 17146 raft_consensus.cc:504] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:09.126453 17146 leader_election.cc:288] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829), 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033)
I20260812 07:11:09.131985 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.126894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:60899 (local address 127.31.156.66:41829)
0812 07:11:09.127064 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:09.127069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:09.127257 (+ 188us) server_negotiation.cc:407] Connection header received
0812 07:11:09.127487 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:09.127490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:09.127548 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:09.127642 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:09.127999 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.128711 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.130422 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.131167 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.131328 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:09.131391 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:09.131788 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 07:11:09.131870 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:11:09.131982 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.127004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:46089 (local address 127.31.156.67:33033)
0812 07:11:09.127143 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:09.127148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:09.127374 (+ 226us) server_negotiation.cc:407] Connection header received
0812 07:11:09.127788 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:09.127792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:09.127846 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:09.127931 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:09.128336 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.129033 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.130456 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.131200 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.131328 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:09.131391 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:09.131737 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 07:11:09.131839 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:11:09.131963 17155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.126779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:41829 (local address 127.31.156.65:60899)
0812 07:11:09.127128 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:09.127146 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:09.127347 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:09.127659 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:09.127662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:09.127679 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:09.127814 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:09.127821 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.128858 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.128862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:09.130228 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:09.130240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.131405 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.131408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:09.131497 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:09.131524 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 07:11:09.131621 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:11:09.131757 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":125,"thread_start_us":83,"threads_started":1}
I20260812 07:11:09.131963 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.126857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:33033 (local address 127.31.156.65:46089)
0812 07:11:09.127261 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:09.127285 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:09.127386 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:09.127999 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:09.128003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:09.128026 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:09.128142 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:09.128149 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.129165 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.129168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:09.130252 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:09.130264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.131329 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.131333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:09.131449 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:09.131524 (+ 75us) client_negotiation.cc:769] Sending connection context
0812 07:11:09.131621 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:11:09.131757 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":77,"thread_start_us":125,"threads_started":1}
I20260812 07:11:09.132817 16931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ec41199977468c9e6c850388011ffd" candidate_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28668e338ddd4aacaacba83a4f86fbb5" is_pre_election: true
I20260812 07:11:09.133225 16931 raft_consensus.cc:2454] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89bf44c8f45e49ac8d4eb7129cfdcca2 in term 0.
I20260812 07:11:09.133262 17077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ec41199977468c9e6c850388011ffd" candidate_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d0b849885f54ee4803723f30e57028d" is_pre_election: true
I20260812 07:11:09.133543 17077 raft_consensus.cc:2454] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89bf44c8f45e49ac8d4eb7129cfdcca2 in term 0.
I20260812 07:11:09.133826 16708 leader_election.cc:302] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28668e338ddd4aacaacba83a4f86fbb5, 89bf44c8f45e49ac8d4eb7129cfdcca2; no voters:
I20260812 07:11:09.134279 17146 raft_consensus.cc:2781] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:11:09.134408 17146 raft_consensus.cc:482] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:11:09.134497 17146 raft_consensus.cc:3037] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:09.135838 17146 raft_consensus.cc:504] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:09.136318 17146 leader_election.cc:288] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 1 election: Requested vote from peers 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829), 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033)
I20260812 07:11:09.136842 16931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ec41199977468c9e6c850388011ffd" candidate_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28668e338ddd4aacaacba83a4f86fbb5"
I20260812 07:11:09.136837 17077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ec41199977468c9e6c850388011ffd" candidate_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d0b849885f54ee4803723f30e57028d"
I20260812 07:11:09.137002 16931 raft_consensus.cc:3037] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:09.137002 17077 raft_consensus.cc:3037] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:09.138602 17077 raft_consensus.cc:2454] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89bf44c8f45e49ac8d4eb7129cfdcca2 in term 1.
I20260812 07:11:09.138784 16931 raft_consensus.cc:2454] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89bf44c8f45e49ac8d4eb7129cfdcca2 in term 1.
I20260812 07:11:09.139153 16709 leader_election.cc:302] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 28668e338ddd4aacaacba83a4f86fbb5, 89bf44c8f45e49ac8d4eb7129cfdcca2, 9d0b849885f54ee4803723f30e57028d; no voters:
I20260812 07:11:09.139436 17146 raft_consensus.cc:2781] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:11:09.139737 17146 raft_consensus.cc:686] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 LEADER]: Becoming Leader. State: Replica: 89bf44c8f45e49ac8d4eb7129cfdcca2, State: Running, Role: LEADER
I20260812 07:11:09.140064 17146 consensus_queue.cc:237] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:09.143699 16626 catalog_manager.cc:5495] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89bf44c8f45e49ac8d4eb7129cfdcca2 (127.31.156.65). New cstate: current_term: 1 leader_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } health_report { overall_health: HEALTHY } } }
I20260812 07:11:09.177361 17160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.172956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53548 (local address 127.31.156.65:34273)
0812 07:11:09.173272 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:09.173287 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:09.173303 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:11:09.173392 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:09.173395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:09.173452 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:09.173595 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:11:09.173860 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.174415 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.175471 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:09.175713 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:09.175907 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:09.176189 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:11:09.176681 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:11:09.176826 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:09.176893 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 07:11:09.176958 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":94,"threads_started":1}
I20260812 07:11:09.182420 16931 raft_consensus.cc:1261] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5 [term 1 FOLLOWER]: Refusing update from remote peer 89bf44c8f45e49ac8d4eb7129cfdcca2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:09.182466 17077 raft_consensus.cc:1261] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d [term 1 FOLLOWER]: Refusing update from remote peer 89bf44c8f45e49ac8d4eb7129cfdcca2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:09.183346 17157 consensus_queue.cc:1035] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:09.183671 17146 consensus_queue.cc:1035] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:09.194442 17162 mvcc.cc:205] Tried to move back new op lower bound from 7317580468963561472 to 7317580468800692224. Current Snapshot: MvccSnapshot[applied={T|T < 7317580468963561472}]
I20260812 07:11:09.196628 17161 mvcc.cc:205] Tried to move back new op lower bound from 7317580468963561472 to 7317580468800692224. Current Snapshot: MvccSnapshot[applied={T|T < 7317580468963561472 or (T in {7317580468963561472})}]
I20260812 07:11:09.204582 16625 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:11:09.204809 16625 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:11:09.208060 16625 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Sending DeleteTablet for 3 replicas of tablet 49ec41199977468c9e6c850388011ffd
I20260812 07:11:09.209045 16910 tablet_service.cc:1514] Processing DeleteTablet for tablet 49ec41199977468c9e6c850388011ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:09 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:38402
I20260812 07:11:09.209741 17171 tablet_replica.cc:330] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: stopping tablet replica
I20260812 07:11:09.209753 17053 tablet_service.cc:1514] Processing DeleteTablet for tablet 49ec41199977468c9e6c850388011ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:09 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:34490
I20260812 07:11:09.210085 16761 tablet_service.cc:1514] Processing DeleteTablet for tablet 49ec41199977468c9e6c850388011ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:09 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:53546
I20260812 07:11:09.210343 17171 raft_consensus.cc:2229] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:11:09.210562 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 16668
I20260812 07:11:09.210564 17172 tablet_replica.cc:330] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d: stopping tablet replica
I20260812 07:11:09.212862 17172 raft_consensus.cc:2229] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:11:09.214794 17172 raft_consensus.cc:2258] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:09.215342 17171 raft_consensus.cc:2258] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:09.217296 17171 ts_tablet_manager.cc:1905] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:11:09.220124 17172 ts_tablet_manager.cc:1905] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 07:11:09.223033 16609 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:34273 (error 108)
I20260812 07:11:09.223461 17172 ts_tablet_manager.cc:1918] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:09.223592 17172 log.cc:1196] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/49ec41199977468c9e6c850388011ffd
I20260812 07:11:09.224040 17172 ts_tablet_manager.cc:1939] T 49ec41199977468c9e6c850388011ffd P 9d0b849885f54ee4803723f30e57028d: Deleting consensus metadata
I20260812 07:11:09.224059 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 16836
I20260812 07:11:09.227393 16612 catalog_manager.cc:4841] TS 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033): tablet 49ec41199977468c9e6c850388011ffd (table test-table [id=a55443de81ba4140a81b8f8ee465ee43]) successfully deleted
I20260812 07:11:09.231988 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.231602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34273 (local address 127.0.0.1:53550)
0812 07:11:09.231785 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:09.231881 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260812 07:11:09.232376 16609 catalog_manager.cc:4584] TS 89bf44c8f45e49ac8d4eb7129cfdcca2 (127.31.156.65:34273): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 49ec41199977468c9e6c850388011ffd: Network error: Client connection negotiation failed: client connection to 127.31.156.65:34273: connect: Connection refused (error 111)
I20260812 07:11:09.234850 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 16985
I20260812 07:11:09.239310 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.235055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:41829 (local address 127.0.0.1:38412)
0812 07:11:09.239087 (+ 4032us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:09.239183 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.66:41829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3886}
I20260812 07:11:09.245990 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 16587
I20260812 07:11:09.259516 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46033
--webserver_interface=127.31.156.126
--webserver_port=34173
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:46033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:09.383898 17175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:09.384238 17175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:09.384282 17175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:09.388602 17175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:11:09.388687 17175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:09.388710 17175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:09.388727 17175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:11:09.388743 17175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:11:09.394704 17175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:46033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=34173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:09.395991 17175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:09.397125 17175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:09.406962 17183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:09.406908 17181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:09.406949 17180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:09.407210 17175 server_base.cc:1034] running on GCE node
I20260812 07:11:09.408124 17175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:09.409389 17175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:09.410580 17175 hybrid_clock.cc:648] HybridClock initialized: now 1786518669410566 us; error 36 us; skew 500 ppm
Aug 12 07:11:09 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518669, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:09.413878 17175 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:11:09.415028 17175 webserver.cc:460] Webserver started at http://127.31.156.126:34173/ using document root <none> and password file <none>
I20260812 07:11:09.415670 17175 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:09.415750 17175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:09.420331 17175 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:11:09.423074 17190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:09.424571 17175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:09.424746 17175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "ad7107e2ce0e4c749ebf6ab04768b72f"
format_stamp: "Formatted at 2026-08-12 07:11:06 on dist-test-slave-csg5"
server_key: "03a8481afce765a266bd1eb896a37093"
server_key_iv: "0a9b4920c4388dc5ed405bf77fd9ae41"
server_key_version: "encryptionkey@0"
I20260812 07:11:09.425458 17175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:09.447045 17175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:09.450798 17175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:09.451040 17175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:09.459295 17175 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:46033
I20260812 07:11:09.459676 17251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:46033 every 8 connection(s)
I20260812 07:11:09.460583 17175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:11:09.465977 17252 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:11:09.467995 17252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Bootstrap starting.
I20260812 07:11:09.468848 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 17175
I20260812 07:11:09.469362 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:34273
--local_ip_for_outbound_sockets=127.31.156.65
--tserver_master_addrs=127.31.156.126:46033
--webserver_port=35421
--webserver_interface=127.31.156.65
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:11:09.497188 17252 log.cc:824] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:09.514214 17252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:11:09.514845 17252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: Bootstrap complete.
I20260812 07:11:09.521303 17252 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:09.522008 17252 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad7107e2ce0e4c749ebf6ab04768b72f, State: Initialized, Role: FOLLOWER
I20260812 07:11:09.522696 17252 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:09.522874 17252 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:11:09.522954 17252 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:11:09.523069 17252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 2 FOLLOWER]: Advancing to term 3
I20260812 07:11:09.526422 17252 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:09.526898 17252 leader_election.cc:302] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad7107e2ce0e4c749ebf6ab04768b72f; no voters:
I20260812 07:11:09.527343 17252 leader_election.cc:288] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 07:11:09.527740 17256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 3 FOLLOWER]: Leader election won for term 3
I20260812 07:11:09.528152 17256 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [term 3 LEADER]: Becoming Leader. State: Replica: ad7107e2ce0e4c749ebf6ab04768b72f, State: Running, Role: LEADER
I20260812 07:11:09.528606 17256 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } }
I20260812 07:11:09.528925 17252 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:11:09.530403 17257 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } } }
I20260812 07:11:09.530557 17257 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:09.531029 17258 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad7107e2ce0e4c749ebf6ab04768b72f. Latest consensus state: current_term: 3 leader_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad7107e2ce0e4c749ebf6ab04768b72f" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46033 } } }
I20260812 07:11:09.531217 17258 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:09.535439 17272 catalog_manager.cc:1259] Loaded cluster ID: a390eadcc7f5481ebd5149f3bdc2b774
I20260812 07:11:09.535539 17272 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: loading cluster ID for follower catalog manager: success
W20260812 07:11:09.536866 17272 catalog_manager.cc:1576] T 00000000000000000000000000000000 P ad7107e2ce0e4c749ebf6ab04768b72f: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260812 07:11:09.537559 17272 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260812 07:11:09.537719 17266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:11:09.542084 17266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:11:09.542899 17266 catalog_manager.cc:1259] Loaded cluster ID: a390eadcc7f5481ebd5149f3bdc2b774
I20260812 07:11:09.542984 17266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 07:11:09.543861 17266 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f711362d9ad google::LogMessage::Fail() at ??:0
@ 0x7f711363088a google::LogMessage::SendToLog() at ??:0
@ 0x7f711362d4de google::LogMessage::Flush() at ??:0
@ 0x7f711362ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f711df6cfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f711df6dd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f711df6a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f711dfae640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f711587f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f711587fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f71158813f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f7115870512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7116eb46db start_thread at ??:0
@ 0x7f711212b61f clone at ??:0
W20260812 07:11:09.663968 17254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:09.664307 17254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:09.664369 17254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:09.669646 17254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:09.669761 17254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:09.669912 17254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:11:09.681890 17254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:34273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=35421
--tserver_master_addrs=127.31.156.126:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:09.683776 17254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:09.685367 17254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:09.699095 17297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:09.699627 17294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:09.699630 17254 server_base.cc:1034] running on GCE node
W20260812 07:11:09.701884 17295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:09.702757 17254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:09.704048 17254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:09.705308 17254 hybrid_clock.cc:648] HybridClock initialized: now 1786518669705269 us; error 67 us; skew 500 ppm
Aug 12 07:11:09 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518669, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:09.710032 17254 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:11:09.711762 17254 webserver.cc:460] Webserver started at http://127.31.156.65:35421/ using document root <none> and password file <none>
I20260812 07:11:09.712703 17254 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:09.712834 17254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:09.720039 17254 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260812 07:11:09.723819 17304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:09.726106 17254 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:11:09.726337 17254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "282de14ccab1062e50213a4bdd19c17b"
server_key_iv: "ac4a9d4b13cc76b7ee3d0b3ac8931354"
server_key_version: "encryptionkey@0"
I20260812 07:11:09.727391 17254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:09.746054 17254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:09.750361 17254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:09.750638 17254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:09.751421 17254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:09.752660 17311 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 07:11:09.757898 17254 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 07:11:09.758013 17254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260812 07:11:09.758082 17254 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 07:11:09.760413 17254 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 07:11:09.760489 17254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260812 07:11:09.760962 17311 tablet_bootstrap.cc:492] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2: Bootstrap starting.
I20260812 07:11:09.772670 17254 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:34273
I20260812 07:11:09.772797 17438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:34273 every 8 connection(s)
I20260812 07:11:09.773882 17254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:11:09.781957 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 17254
I20260812 07:11:09.782630 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:41829
--local_ip_for_outbound_sockets=127.31.156.66
--tserver_master_addrs=127.31.156.126:46033
--webserver_port=36979
--webserver_interface=127.31.156.66
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:11:09.785501 17311 log.cc:824] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:09.796986 17311 tablet_bootstrap.cc:492] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:11:09.797575 17311 tablet_bootstrap.cc:492] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2: Bootstrap complete.
I20260812 07:11:09.799561 17311 ts_tablet_manager.cc:1397] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2: Time spent bootstrapping tablet: real 0.039s user 0.025s sys 0.013s
I20260812 07:11:09.803879 17311 raft_consensus.cc:348] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:09.804638 17311 raft_consensus.cc:729] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf44c8f45e49ac8d4eb7129cfdcca2, State: Initialized, Role: FOLLOWER
I20260812 07:11:09.805382 17311 consensus_queue.cc:260] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:09.806898 17311 ts_tablet_manager.cc:1428] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
W20260812 07:11:09.852538 17441 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:11:09.776173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:60007)
0812 07:11:09.776788 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:09.776863 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:09.777805 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:09.851671 (+ 73866us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":175,"threads_started":1}
W20260812 07:11:09.853364 17439 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:46033 (0 consecutive failures): Network error: Failed to ping master at 127.31.156.126:46033: Client connection negotiation failed: client connection to 127.31.156.126:46033: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 07:11:09.854326 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.853982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:39007)
0812 07:11:09.854147 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:09.854246 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 07:11:09.855242 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:09.854981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:58523)
0812 07:11:09.855111 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:09.855169 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 07:11:09.855515 17439 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:11:09.925715 17442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:09.926067 17442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:09.926117 17442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:09.930552 17442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:09.930678 17442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:09.930747 17442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:11:09.937080 17442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:41829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=36979
--tserver_master_addrs=127.31.156.126:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:09.938365 17442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:09.939597 17442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:09.951884 17450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:09.952030 17449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:09.951911 17452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:09.952242 17442 server_base.cc:1034] running on GCE node
I20260812 07:11:09.952881 17442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:09.953667 17442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:09.954922 17442 hybrid_clock.cc:648] HybridClock initialized: now 1786518669954903 us; error 36 us; skew 500 ppm
Aug 12 07:11:09 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518669, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:09.958565 17442 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:11:09.959748 17442 webserver.cc:460] Webserver started at http://127.31.156.66:36979/ using document root <none> and password file <none>
I20260812 07:11:09.960382 17442 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:09.960485 17442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:09.965016 17442 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:11:09.968374 17460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:09.970019 17442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:11:09.970218 17442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "28668e338ddd4aacaacba83a4f86fbb5"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "22597b2c07b93c092763500ce89a25bc"
server_key_iv: "771ee5778821402c8e76df107f505e55"
server_key_version: "encryptionkey@0"
I20260812 07:11:09.970978 17442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:09.998034 17442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:10.002763 17442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:10.003299 17442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:10.004177 17442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:10.006500 17468 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260812 07:11:10.012532 17468 ts_tablet_manager.cc:1852] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260812 07:11:10.013068 17468 ts_tablet_manager.cc:1905] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:11:10.018546 17468 ts_tablet_manager.cc:1918] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:10.018937 17468 log.cc:1196] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/49ec41199977468c9e6c850388011ffd
I20260812 07:11:10.019460 17468 ts_tablet_manager.cc:1939] T 49ec41199977468c9e6c850388011ffd P 28668e338ddd4aacaacba83a4f86fbb5: Deleting consensus metadata
I20260812 07:11:10.020093 17442 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 07:11:10.020195 17442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.014s user 0.001s sys 0.000s
I20260812 07:11:10.020310 17442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:10.020350 17442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:10.034502 17442 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:41829
I20260812 07:11:10.034729 17589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:41829 every 8 connection(s)
I20260812 07:11:10.035676 17442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:11:10.039973 17592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:10.038379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:53277)
0812 07:11:10.039040 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:10.039179 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":216,"threads_started":1}
W20260812 07:11:10.040541 17590 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:46033 (0 consecutive failures): Network error: Failed to ping master at 127.31.156.126:46033: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
I20260812 07:11:10.041397 17592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:10.041110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:35485)
0812 07:11:10.041262 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:10.041327 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 07:11:10.042255 17592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:10.042007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:43007)
0812 07:11:10.042142 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:10.042198 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 07:11:10.042582 17590 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:11:10.044481 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 17442
I20260812 07:11:10.045034 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:33033
--local_ip_for_outbound_sockets=127.31.156.67
--tserver_master_addrs=127.31.156.126:46033
--webserver_port=40987
--webserver_interface=127.31.156.67
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:10.184949 17593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:10.185410 17593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:10.185511 17593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:10.190780 17593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:10.190912 17593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:10.191027 17593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:11:10.197901 17593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:33033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=40987
--tserver_master_addrs=127.31.156.126:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:10.199554 17593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:10.200853 17593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:10.213267 17598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:10.213287 17601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:10.213410 17599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:10.213347 17593 server_base.cc:1034] running on GCE node
I20260812 07:11:10.214224 17593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:10.215097 17593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:10.216331 17593 hybrid_clock.cc:648] HybridClock initialized: now 1786518670216307 us; error 40 us; skew 500 ppm
Aug 12 07:11:10 dist-test-slave-csg5 krb5kdc[15979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518670, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:10.233060 17593 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:11:10.234539 17593 webserver.cc:460] Webserver started at http://127.31.156.67:40987/ using document root <none> and password file <none>
I20260812 07:11:10.235317 17593 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:10.235435 17593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:10.240456 17593 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:11:10.243322 17608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:10.245087 17593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:10.245319 17593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "9d0b849885f54ee4803723f30e57028d"
format_stamp: "Formatted at 2026-08-12 07:11:07 on dist-test-slave-csg5"
server_key: "88d90f39ddba88efc8bd734904dd736d"
server_key_iv: "1a2445d74b407b64a4f3d655af44b808"
server_key_version: "encryptionkey@0"
I20260812 07:11:10.246008 17593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:10.269966 17593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:10.274462 17593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:10.274787 17593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:10.275575 17593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:10.276636 17593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:10.276744 17593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:10.276840 17593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:10.276881 17593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:10.289423 17593 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:33033
I20260812 07:11:10.289449 17733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:33033 every 8 connection(s)
I20260812 07:11:10.290870 17593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:11:10.296401 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 17593
I20260812 07:11:10.297053 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:10.294027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:46379)
0812 07:11:10.294932 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:10.295455 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":381,"threads_started":1}
W20260812 07:11:10.298367 17734 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:46033 (0 consecutive failures): Network error: Failed to ping master at 127.31.156.126:46033: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
I20260812 07:11:10.299285 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:10.299004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:60239)
0812 07:11:10.299150 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:10.299209 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 07:11:10.300186 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:10.299894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:42269)
0812 07:11:10.300056 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:10.300113 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260812 07:11:10.300558 17734 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:11:10.856724 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:10.856224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:44573)
0812 07:11:10.856506 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:10.856589 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":78,"threads_started":1}
I20260812 07:11:11.044045 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:11.043519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:44319)
0812 07:11:11.043845 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:11.043929 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":84,"threads_started":1}
I20260812 07:11:11.302057 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:11.301466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:32877)
0812 07:11:11.301850 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:11.301923 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":90,"threads_started":1}
I20260812 07:11:11.429920 17743 raft_consensus.cc:482] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:11:11.430207 17743 raft_consensus.cc:504] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:11.431627 17743 leader_election.cc:288] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829), 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033)
Aug 12 07:11:11 dist-test-slave-csg5 krb5kdc[15979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518669, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.67@KRBTEST.COM
Aug 12 07:11:11 dist-test-slave-csg5 krb5kdc[15979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518669, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.66@KRBTEST.COM
I20260812 07:11:11.448738 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:11.432311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:53717 (local address 127.31.156.66:41829)
0812 07:11:11.432977 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:11.432993 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:11.433044 (+ 51us) server_negotiation.cc:407] Connection header received
0812 07:11:11.433987 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:11.433995 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:11.434410 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:11.434772 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:11.436164 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:11.437395 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:11.438721 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:11.439257 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:11.439473 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:11.443081 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:11.443086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:11.443100 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:11.443135 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:11.445454 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:11.446288 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:11.446295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:11.446302 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:11.446450 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:11.446939 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:11.446942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:11.446944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:11.447432 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:11.447594 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:11.448211 (+ 617us) server_negotiation.cc:299] Negotiation successful
0812 07:11:11.448507 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":126,"threads_started":1}
I20260812 07:11:11.448725 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:11.431961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:41829 (local address 127.31.156.65:53717)
0812 07:11:11.432389 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:11.432409 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:11.432536 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:11.434725 (+ 2189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:11.434733 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:11.435369 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:11.435946 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:11.435969 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:11.437549 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:11.437555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:11.438533 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:11.438546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:11.439422 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:11.439428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:11.439678 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:11.440418 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:11.440438 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:11.442835 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:11.445643 (+ 2808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:11.445651 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:11.445669 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:11.446131 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:11.446565 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:11.446569 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:11.446570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:11.446831 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:11.447567 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:11.447574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:11.447855 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 07:11:11.448063 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 07:11:11.448516 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":166,"spinlock_wait_cycles":31616,"thread_start_us":110,"threads_started":1}
I20260812 07:11:11.448748 17745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:11.432168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:59569 (local address 127.31.156.67:33033)
0812 07:11:11.432856 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:11.432886 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:11.433031 (+ 145us) server_negotiation.cc:407] Connection header received
0812 07:11:11.433828 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:11.433835 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:11.434229 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:11.434608 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:11.436173 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:11.437342 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:11.438699 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:11.439252 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:11.439460 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:11.443022 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:11.443028 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:11.443086 (+ 58us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:11.443132 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:11.445524 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:11.446387 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:11.446392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:11.446397 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:11.446538 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:11.446947 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:11.446950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:11.446952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:11.447433 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:11.447594 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:11.448215 (+ 621us) server_negotiation.cc:299] Negotiation successful
0812 07:11:11.448539 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":100,"threads_started":1}
I20260812 07:11:11.448792 17747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:11.431961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:33033 (local address 127.31.156.65:59569)
0812 07:11:11.432873 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:11.432888 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:11.433004 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:11.434725 (+ 1721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:11.434732 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:11.435418 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:11.435946 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:11.435968 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:11.437548 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:11.437551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:11.438506 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:11.438519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:11.439421 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:11.439425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:11.439669 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:11.440471 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:11.440484 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:11.442574 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:11.445684 (+ 3110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:11.445689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:11.445691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:11.446146 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:11.446678 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:11.446683 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:11.446685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:11.446829 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:11.447567 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:11.447574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:11.447855 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 07:11:11.448061 (+ 206us) client_negotiation.cc:240] Negotiation successful
0812 07:11:11.448615 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":907,"spinlock_wait_cycles":40320,"thread_start_us":86,"threads_started":1}
W20260812 07:11:11.451879 17307 leader_election.cc:341] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829): Not found: Tablet not found: 49ec41199977468c9e6c850388011ffd
W20260812 07:11:11.452134 17308 leader_election.cc:341] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033): Not found: Tablet not found: 49ec41199977468c9e6c850388011ffd
I20260812 07:11:11.452246 17308 leader_election.cc:302] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89bf44c8f45e49ac8d4eb7129cfdcca2; no voters: 28668e338ddd4aacaacba83a4f86fbb5, 9d0b849885f54ee4803723f30e57028d
I20260812 07:11:11.452549 17743 raft_consensus.cc:2726] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:11:11.858270 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:11.857781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:36913)
0812 07:11:11.858072 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:11.858158 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
I20260812 07:11:12.045663 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:12.045032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:50865)
0812 07:11:12.045464 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:12.045538 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":199,"threads_started":1}
I20260812 07:11:12.304065 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:12.303459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:46009)
0812 07:11:12.303865 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:12.303939 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":89,"threads_started":1}
I20260812 07:11:12.860088 17750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:12.859521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:51391)
0812 07:11:12.859900 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:12.859972 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":130,"threads_started":1}
I20260812 07:11:13.047472 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:13.046959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:48625)
0812 07:11:13.047275 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:13.047363 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":97,"threads_started":1}
I20260812 07:11:13.305962 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:13.305423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:47277)
0812 07:11:13.305761 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:13.305834 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":88,"threads_started":1}
I20260812 07:11:13.521054 17754 raft_consensus.cc:482] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:11:13.521242 17754 raft_consensus.cc:504] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:13.521780 17754 leader_election.cc:288] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829), 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033)
W20260812 07:11:13.522956 17307 leader_election.cc:341] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829): Not found: Tablet not found: 49ec41199977468c9e6c850388011ffd
W20260812 07:11:13.523061 17308 leader_election.cc:341] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033): Not found: Tablet not found: 49ec41199977468c9e6c850388011ffd
I20260812 07:11:13.523108 17307 leader_election.cc:302] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89bf44c8f45e49ac8d4eb7129cfdcca2; no voters: 28668e338ddd4aacaacba83a4f86fbb5, 9d0b849885f54ee4803723f30e57028d
I20260812 07:11:13.523275 17754 raft_consensus.cc:2726] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:11:13.861685 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:13.861149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:53073)
0812 07:11:13.861497 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:13.861567 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":94,"threads_started":1}
I20260812 07:11:14.049155 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:14.048516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:44931)
0812 07:11:14.048896 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:14.048989 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":90,"threads_started":1}
I20260812 07:11:14.307854 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:14.307260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:51791)
0812 07:11:14.307666 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:14.307739 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":147,"threads_started":1}
I20260812 07:11:14.863759 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:14.863047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:56321)
0812 07:11:14.863498 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:14.863583 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":108,"threads_started":1}
I20260812 07:11:15.050859 17759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:15.050169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:53927)
0812 07:11:15.050562 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:15.050731 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":147,"threads_started":1}
I20260812 07:11:15.309468 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:15.308878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:52891)
0812 07:11:15.309213 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:15.309282 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260812 07:11:15.865326 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:15.864732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:60213)
0812 07:11:15.865131 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:15.865212 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":127,"threads_started":1}
I20260812 07:11:16.006860 17762 raft_consensus.cc:482] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:11:16.007022 17762 raft_consensus.cc:504] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:16.007653 17762 leader_election.cc:288] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829), 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033)
W20260812 07:11:16.008679 17307 leader_election.cc:341] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829): Not found: Tablet not found: 49ec41199977468c9e6c850388011ffd
W20260812 07:11:16.008896 17308 leader_election.cc:341] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033): Not found: Tablet not found: 49ec41199977468c9e6c850388011ffd
I20260812 07:11:16.008980 17308 leader_election.cc:302] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89bf44c8f45e49ac8d4eb7129cfdcca2; no voters: 28668e338ddd4aacaacba83a4f86fbb5, 9d0b849885f54ee4803723f30e57028d
I20260812 07:11:16.009224 17762 raft_consensus.cc:2726] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:11:16.052367 17763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:16.051797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:55065)
0812 07:11:16.052174 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:16.052246 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":86,"threads_started":1}
I20260812 07:11:16.311348 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:16.310588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:55103)
0812 07:11:16.311047 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:16.311121 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":206,"threads_started":1}
I20260812 07:11:16.867218 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:16.866711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:59331)
0812 07:11:16.867041 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:16.867112 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":131,"threads_started":1}
I20260812 07:11:17.054064 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:17.053529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:38901)
0812 07:11:17.053890 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:17.053957 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":83,"threads_started":1}
I20260812 07:11:17.313009 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:17.312415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:43687)
0812 07:11:17.312822 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:17.312894 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":93,"threads_started":1}
I20260812 07:11:17.868975 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:17.868387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:34891)
0812 07:11:17.868774 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:17.868862 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":89,"threads_started":1}
I20260812 07:11:18.055819 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:18.055200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:45287)
0812 07:11:18.055585 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:18.055651 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":98,"threads_started":1}
I20260812 07:11:18.300359 17772 raft_consensus.cc:482] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:11:18.300544 17772 raft_consensus.cc:504] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:18.301095 17772 leader_election.cc:288] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829), 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033)
W20260812 07:11:18.301991 17307 leader_election.cc:341] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829): Not found: Tablet not found: 49ec41199977468c9e6c850388011ffd
W20260812 07:11:18.302101 17308 leader_election.cc:341] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033): Not found: Tablet not found: 49ec41199977468c9e6c850388011ffd
I20260812 07:11:18.302233 17308 leader_election.cc:302] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89bf44c8f45e49ac8d4eb7129cfdcca2; no voters: 28668e338ddd4aacaacba83a4f86fbb5, 9d0b849885f54ee4803723f30e57028d
I20260812 07:11:18.302397 17772 raft_consensus.cc:2726] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:11:18.314507 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:18.314014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:34135)
0812 07:11:18.314330 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:18.314398 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":105,"threads_started":1}
I20260812 07:11:18.870968 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:18.870213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:52203)
0812 07:11:18.870622 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:18.870852 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":212,"threads_started":1}
I20260812 07:11:19.057619 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:19.057014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:59789)
0812 07:11:19.057440 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:19.057507 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":121,"threads_started":1}
I20260812 07:11:19.316184 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:19.315606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:55879)
0812 07:11:19.315994 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:19.316069 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":98,"threads_started":1}
I20260812 07:11:19.872771 17777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:19.872281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:54477)
0812 07:11:19.872585 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:19.872655 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":93,"threads_started":1}
I20260812 07:11:20.059265 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:20.058645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:59103)
0812 07:11:20.059055 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:20.059132 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":87,"threads_started":1}
I20260812 07:11:20.317876 17779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:20.317371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:38881)
0812 07:11:20.317691 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:20.317760 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":87,"threads_started":1}
I20260812 07:11:20.874472 17780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:20.873871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:48509)
0812 07:11:20.874227 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:20.874296 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":111,"threads_started":1}
I20260812 07:11:21.060997 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:21.060397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:59873)
0812 07:11:21.060808 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:21.060882 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":89,"threads_started":1}
I20260812 07:11:21.319417 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:21.318865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:59507)
0812 07:11:21.319190 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:21.319270 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":84,"threads_started":1}
I20260812 07:11:21.876097 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:21.875542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:46413)
0812 07:11:21.875908 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:21.875983 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":89,"threads_started":1}
I20260812 07:11:22.062589 17784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:22.062078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:35233)
0812 07:11:22.062417 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:22.062484 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":101,"threads_started":1}
I20260812 07:11:22.320889 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:22.320310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:39483)
0812 07:11:22.320684 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:22.320751 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":125,"threads_started":1}
I20260812 07:11:22.877899 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:22.877385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:46407)
0812 07:11:22.877723 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:22.877792 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":87,"threads_started":1}
I20260812 07:11:23.064072 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:23.063547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:57321)
0812 07:11:23.063860 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:23.063939 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":121,"threads_started":1}
I20260812 07:11:23.322471 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:23.321961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:46613)
0812 07:11:23.322287 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:23.322360 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":86,"threads_started":1}
I20260812 07:11:23.879519 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:23.878977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:40195)
0812 07:11:23.879320 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:23.879402 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":107,"threads_started":1}
I20260812 07:11:24.065647 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:24.065025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:53383)
0812 07:11:24.065475 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:24.065543 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":150,"threads_started":1}
I20260812 07:11:24.265075 17791 raft_consensus.cc:482] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:11:24.265422 17791 raft_consensus.cc:504] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28668e338ddd4aacaacba83a4f86fbb5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41829 } } peers { permanent_uuid: "9d0b849885f54ee4803723f30e57028d" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 33033 } } peers { permanent_uuid: "89bf44c8f45e49ac8d4eb7129cfdcca2" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34273 } }
I20260812 07:11:24.266038 17791 leader_election.cc:288] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829), 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033)
W20260812 07:11:24.267066 17307 leader_election.cc:341] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28668e338ddd4aacaacba83a4f86fbb5 (127.31.156.66:41829): Not found: Tablet not found: 49ec41199977468c9e6c850388011ffd
W20260812 07:11:24.267230 17308 leader_election.cc:341] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9d0b849885f54ee4803723f30e57028d (127.31.156.67:33033): Not found: Tablet not found: 49ec41199977468c9e6c850388011ffd
I20260812 07:11:24.267315 17308 leader_election.cc:302] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89bf44c8f45e49ac8d4eb7129cfdcca2; no voters: 28668e338ddd4aacaacba83a4f86fbb5, 9d0b849885f54ee4803723f30e57028d
I20260812 07:11:24.267639 17791 raft_consensus.cc:2726] T 49ec41199977468c9e6c850388011ffd P 89bf44c8f45e49ac8d4eb7129cfdcca2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:11:24.324148 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:24.323598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.67:51073)
0812 07:11:24.323970 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:24.324036 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":134,"threads_started":1}
I20260812 07:11:24.881600 17793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:24.880898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.65:36789)
0812 07:11:24.881268 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:24.881491 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":119,"threads_started":1}
I20260812 07:11:25.067456 17794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:25.066981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46033 (local address 127.31.156.66:35767)
0812 07:11:25.067277 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:25.067347 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":82,"threads_started":1}
I20260812 07:11:25.298593 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 17254
I20260812 07:11:25.312346 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 17442
I20260812 07:11:25.322906 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 17593
2026-08-12T07:11:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20089 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:11:25 dist-test-slave-csg5 krb5kdc[17798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:11:25 dist-test-slave-csg5 krb5kdc[17798](info): set up 2 sockets
Aug 12 07:11:25 dist-test-slave-csg5 krb5kdc[17798](info): commencing operation
krb5kdc: starting...
W20260812 07:11:26.525594 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:11:26 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:11:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:11:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:11:26.684141 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45227
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:45227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:26.807149 17812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:26.807539 17812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:26.807622 17812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:26.812080 17812 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 07:11:26.812193 17812 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 07:11:26.812223 17812 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 07:11:26.812249 17812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:11:26.812391 17812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:26.812463 17812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:26.812505 17812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:11:26.812541 17812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:11:26.818522 17812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:45227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:26.819912 17812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:26.821128 17812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:26.831795 17817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:26.831745 17820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:26.831881 17812 server_base.cc:1034] running on GCE node
W20260812 07:11:26.831777 17818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:26.832923 17812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:26.834313 17812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:26.835551 17812 hybrid_clock.cc:648] HybridClock initialized: now 1786518686835515 us; error 56 us; skew 500 ppm
Aug 12 07:11:26 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518686, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:26.839098 17812 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:11:26.840198 17812 webserver.cc:460] Webserver started at http://127.31.156.126:33903/ using document root <none> and password file <none>
I20260812 07:11:26.840832 17812 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:26.840940 17812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:26.841193 17812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:26.843477 17812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "3194b83909314a679fd00ec5cf818283"
format_stamp: "Formatted at 2026-08-12 07:11:26 on dist-test-slave-csg5"
server_key: "159cadc1cf25499a004c3b2db0700e31"
server_key_iv: "99843a8fbae9ce869c72025319a369c8"
server_key_version: "encryptionkey@0"
I20260812 07:11:26.844113 17812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "3194b83909314a679fd00ec5cf818283"
format_stamp: "Formatted at 2026-08-12 07:11:26 on dist-test-slave-csg5"
server_key: "159cadc1cf25499a004c3b2db0700e31"
server_key_iv: "99843a8fbae9ce869c72025319a369c8"
server_key_version: "encryptionkey@0"
I20260812 07:11:26.848606 17812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:11:26.851959 17827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:26.853655 17812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 07:11:26.853863 17812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "3194b83909314a679fd00ec5cf818283"
format_stamp: "Formatted at 2026-08-12 07:11:26 on dist-test-slave-csg5"
server_key: "159cadc1cf25499a004c3b2db0700e31"
server_key_iv: "99843a8fbae9ce869c72025319a369c8"
server_key_version: "encryptionkey@0"
I20260812 07:11:26.854012 17812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:26.871208 17812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:26.875168 17812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:26.875415 17812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:26.883246 17812 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:45227
I20260812 07:11:26.883606 17884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:45227 every 8 connection(s)
I20260812 07:11:26.884421 17812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:11:26.887655 17885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:26.893872 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 17812
I20260812 07:11:26.893992 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:11:26.894240 17885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Bootstrap starting.
I20260812 07:11:26.894304 32369 external_mini_cluster.cc:1428] Setting key 3fb687ebe50f63b02a6611079a5a241b
I20260812 07:11:26.897294 17885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:26.898152 17885 log.cc:824] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:26.900372 17885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: No bootstrap required, opened a new log
Aug 12 07:11:26 dist-test-slave-csg5 krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:26.904520 17885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:11:26.904824 17885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:26.904904 17885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3194b83909314a679fd00ec5cf818283, State: Initialized, Role: FOLLOWER
I20260812 07:11:26.905498 17885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:11:26.905712 17885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:11:26.905802 17885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:11:26.905911 17885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:26.907112 17885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:11:26.907503 17885 leader_election.cc:302] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3194b83909314a679fd00ec5cf818283; no voters:
I20260812 07:11:26.907922 17885 leader_election.cc:288] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:11:26.908087 17892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:11:26.908514 17892 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 1 LEADER]: Becoming Leader. State: Replica: 3194b83909314a679fd00ec5cf818283, State: Running, Role: LEADER
I20260812 07:11:26.908907 17892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:11:26.909776 17885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:11:26.910769 17892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3194b83909314a679fd00ec5cf818283. Latest consensus state: current_term: 1 leader_uuid: "3194b83909314a679fd00ec5cf818283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } } }
I20260812 07:11:26.910974 17892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:26.911484 17901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:11:26.912074 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:26.895866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.31.156.126:45227)
0812 07:11:26.896489 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:26.896506 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:26.896550 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:11:26.897291 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:26.897311 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:26.897843 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:26.898246 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:26.899273 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:26.900327 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:26.901324 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:26.901833 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:26.902020 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:26.904926 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:26.904936 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:26.904957 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:26.905003 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:26.907533 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:26.908209 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:26.908216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:26.908225 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:26.908410 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:26.908859 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:26.908862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:26.908864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:26.909737 (+ 873us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:26.910007 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:26.910381 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 07:11:26.910792 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":162,"threads_started":1}
I20260812 07:11:26.914209 17893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3194b83909314a679fd00ec5cf818283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } } }
I20260812 07:11:26.914481 17893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:26.915450 17901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:11:26.923760 17901 catalog_manager.cc:1347] Generated new cluster ID: 0c991bcf85f749eb94f523b2235cc174
I20260812 07:11:26.923870 17901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:11:26.940496 17901 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:11:26.942057 17901 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:11:26.956619 17901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Generated new TSK 0
I20260812 07:11:26.957716 17901 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:11:26.998903 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45227
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:27.131181 17914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:27.131527 17914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:27.131577 17914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:27.136684 17914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:27.136765 17914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:27.136860 17914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:11:27.144049 17914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:27.145468 17914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:27.146819 17914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:27.159005 17923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:27.159001 17920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:27.159022 17921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:27.159451 17914 server_base.cc:1034] running on GCE node
I20260812 07:11:27.160096 17914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:27.160848 17914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:27.162047 17914 hybrid_clock.cc:648] HybridClock initialized: now 1786518687162025 us; error 36 us; skew 500 ppm
Aug 12 07:11:27 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518687, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:27.165509 17914 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:11:27.166630 17914 webserver.cc:460] Webserver started at http://127.31.156.65:42499/ using document root <none> and password file <none>
I20260812 07:11:27.167366 17914 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:27.167438 17914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:27.167681 17914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:27.169968 17914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "8fb049cebf9644efabdec069ffd1ba8b"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "6f14d200dcc99d717f6083f976cf49e1"
server_key_iv: "d1e8bed248a2c2026d1fba4a7ccea77b"
server_key_version: "encryptionkey@0"
I20260812 07:11:27.170696 17914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "8fb049cebf9644efabdec069ffd1ba8b"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "6f14d200dcc99d717f6083f976cf49e1"
server_key_iv: "d1e8bed248a2c2026d1fba4a7ccea77b"
server_key_version: "encryptionkey@0"
I20260812 07:11:27.175261 17914 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:11:27.178258 17930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:27.180070 17914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:27.180251 17914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "8fb049cebf9644efabdec069ffd1ba8b"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "6f14d200dcc99d717f6083f976cf49e1"
server_key_iv: "d1e8bed248a2c2026d1fba4a7ccea77b"
server_key_version: "encryptionkey@0"
I20260812 07:11:27.180421 17914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:27.205476 17914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:27.209724 17914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:27.210032 17914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:27.211086 17914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:27.212307 17914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:27.212376 17914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:27.212457 17914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:27.212494 17914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:27.223664 17914 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:35271
I20260812 07:11:27.223922 18053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:35271 every 8 connection(s)
I20260812 07:11:27.224869 17914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:11:27.228411 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 17914
I20260812 07:11:27.228585 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:11:27.229005 32369 external_mini_cluster.cc:1428] Setting key 453ef82af6e3b75b554aa9d35ce563cb
Aug 12 07:11:27 dist-test-slave-csg5 krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518687, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:11:27.243536 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.227521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:42801 (local address 127.31.156.126:45227)
0812 07:11:27.227697 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:27.227703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:27.229404 (+ 1701us) server_negotiation.cc:407] Connection header received
0812 07:11:27.230418 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:27.230422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:27.230484 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:27.230555 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:27.231972 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.232927 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.234905 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.235176 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.235276 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.238913 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:27.238916 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:27.238918 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:27.238952 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:27.240704 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:27.241453 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:27.241459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:27.241466 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:27.241564 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:27.242004 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:27.242007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:27.242009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:27.242236 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:27.242448 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:27.243143 (+ 695us) server_negotiation.cc:299] Negotiation successful
0812 07:11:27.243321 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:11:27.244531 18056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.227963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:42801)
0812 07:11:27.229174 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:27.229247 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:27.230180 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:27.230802 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:27.230809 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:27.231246 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:27.231775 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.231791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.233122 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.233128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.234741 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.234758 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.235466 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.235472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.235733 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.236572 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:27.236602 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:27.238574 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:27.240864 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:27.240872 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:27.240894 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:27.241311 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:27.241676 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:27.241680 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:27.241682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:27.241880 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:27.242445 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:27.242454 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:27.242860 (+ 406us) client_negotiation.cc:769] Sending connection context
0812 07:11:27.243078 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 07:11:27.243489 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"thread_start_us":736,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
I20260812 07:11:27.246608 18054 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45227
I20260812 07:11:27.246968 18054 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:27.247715 18054 heartbeater.cc:511] Master 127.31.156.126:45227 requested a full tablet report, sending...
I20260812 07:11:27.249697 17847 ts_manager.cc:194] Registered new tserver with Master: 8fb049cebf9644efabdec069ffd1ba8b (127.31.156.65:35271)
I20260812 07:11:27.251003 17847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:42801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:11:27.271229 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45227
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:27.395099 18059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:27.395421 18059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:27.395467 18059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:27.399703 18059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:27.399783 18059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:27.399848 18059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:11:27.405602 18059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:27.406929 18059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:27.408102 18059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:27.419106 18066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:27.419087 18068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:27.419083 18065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:27.419407 18059 server_base.cc:1034] running on GCE node
I20260812 07:11:27.419942 18059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:27.420629 18059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:27.421864 18059 hybrid_clock.cc:648] HybridClock initialized: now 1786518687421843 us; error 39 us; skew 500 ppm
Aug 12 07:11:27 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518687, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:27.425390 18059 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:11:27.426508 18059 webserver.cc:460] Webserver started at http://127.31.156.66:38261/ using document root <none> and password file <none>
I20260812 07:11:27.427203 18059 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:27.427309 18059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:27.427562 18059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:27.429919 18059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "ddd8d839b30941f39f757b56296514f8"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "a776cab3e01594e2ca18a31c5d71dbe1"
server_key_iv: "0e29f6cd1cc666aa525182bd1e5d1e06"
server_key_version: "encryptionkey@0"
I20260812 07:11:27.430575 18059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "ddd8d839b30941f39f757b56296514f8"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "a776cab3e01594e2ca18a31c5d71dbe1"
server_key_iv: "0e29f6cd1cc666aa525182bd1e5d1e06"
server_key_version: "encryptionkey@0"
I20260812 07:11:27.434983 18059 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:11:27.437947 18076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:27.439545 18059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:27.439715 18059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "ddd8d839b30941f39f757b56296514f8"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "a776cab3e01594e2ca18a31c5d71dbe1"
server_key_iv: "0e29f6cd1cc666aa525182bd1e5d1e06"
server_key_version: "encryptionkey@0"
I20260812 07:11:27.439865 18059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:27.472441 18059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:27.476171 18059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:27.476423 18059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:27.477088 18059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:27.478013 18059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:27.478086 18059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:27.478168 18059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:27.478219 18059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:27.488894 18059 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:36725
I20260812 07:11:27.488912 18201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:36725 every 8 connection(s)
I20260812 07:11:27.489948 18059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:11:27.490767 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 18059
I20260812 07:11:27.490916 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:11:27.491288 32369 external_mini_cluster.cc:1428] Setting key 8d5ce099ca3fbec8e0328936775bf1cb
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
Aug 12 07:11:27 dist-test-slave-csg5 krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518687, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:27.513470 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.492218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:34185 (local address 127.31.156.126:45227)
0812 07:11:27.492393 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:27.492398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:27.495362 (+ 2964us) server_negotiation.cc:407] Connection header received
0812 07:11:27.496945 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:27.496950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:27.497017 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:27.497111 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:27.498727 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.499346 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.500745 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.501025 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.501162 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.508472 (+ 7310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:27.508477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:27.508479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:27.508528 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:27.510538 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:27.511569 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:27.511573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:27.511575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:27.511671 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:27.512129 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:27.512132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:27.512136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:27.512363 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:27.512481 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:27.513116 (+ 635us) server_negotiation.cc:299] Negotiation successful
0812 07:11:27.513283 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:11:27.514485 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.492743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:34185)
0812 07:11:27.495055 (+ 2312us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:27.495217 (+ 162us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:27.496666 (+ 1449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:27.497333 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:27.497342 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:27.497794 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:27.498505 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.498525 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.499511 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.499522 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.500587 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.500599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.501230 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.501234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.501444 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.502305 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:27.502324 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:27.508012 (+ 5688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:27.510790 (+ 2778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:27.510802 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:27.510830 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:27.511389 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:27.511816 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:27.511819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:27.511821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:27.512010 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:27.512531 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:27.512538 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:27.512846 (+ 308us) client_negotiation.cc:769] Sending connection context
0812 07:11:27.513058 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 07:11:27.513468 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2077,"thread_start_us":204,"threads_started":1}
I20260812 07:11:27.515971 18202 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45227
I20260812 07:11:27.516310 18202 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:27.517125 18202 heartbeater.cc:511] Master 127.31.156.126:45227 requested a full tablet report, sending...
I20260812 07:11:27.518747 17847 ts_manager.cc:194] Registered new tserver with Master: ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725)
I20260812 07:11:27.519464 17847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:34185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:11:27.531666 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45227
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:27.670637 18207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:27.671010 18207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:27.671060 18207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:27.675356 18207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:27.675441 18207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:27.675506 18207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:11:27.681403 18207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:45227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:27.682588 18207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:27.683761 18207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:27.695573 18213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:27.695619 18214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:27.695601 18216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:27.696414 18207 server_base.cc:1034] running on GCE node
I20260812 07:11:27.696913 18207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:27.697750 18207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:27.698968 18207 hybrid_clock.cc:648] HybridClock initialized: now 1786518687698927 us; error 61 us; skew 500 ppm
Aug 12 07:11:27 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518687, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:27.702524 18207 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:11:27.703652 18207 webserver.cc:460] Webserver started at http://127.31.156.67:43561/ using document root <none> and password file <none>
I20260812 07:11:27.704286 18207 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:27.704391 18207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:27.704648 18207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:11:27.707360 18207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "1057199d80f447ce9013ca00e6758b81"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "dde7b80e2d3741d7fceca89a7fa41676"
server_key_iv: "bdcd00831e56605d752eb8f9d9e66d79"
server_key_version: "encryptionkey@0"
I20260812 07:11:27.708050 18207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "1057199d80f447ce9013ca00e6758b81"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "dde7b80e2d3741d7fceca89a7fa41676"
server_key_iv: "bdcd00831e56605d752eb8f9d9e66d79"
server_key_version: "encryptionkey@0"
I20260812 07:11:27.713001 18207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:11:27.716331 18223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:27.718030 18207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:11:27.718248 18207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "1057199d80f447ce9013ca00e6758b81"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "dde7b80e2d3741d7fceca89a7fa41676"
server_key_iv: "bdcd00831e56605d752eb8f9d9e66d79"
server_key_version: "encryptionkey@0"
I20260812 07:11:27.718415 18207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:27.740849 18207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:27.744947 18207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:27.745222 18207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:27.745957 18207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:27.747041 18207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:27.747099 18207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:27.747148 18207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:27.747164 18207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:27.757570 18207 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:42863
I20260812 07:11:27.757599 18350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:42863 every 8 connection(s)
I20260812 07:11:27.758742 18207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:11:27.762526 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 18207
I20260812 07:11:27.762725 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:11:27.763095 32369 external_mini_cluster.cc:1428] Setting key f7cd9224071d6bfdd6c682b0558e3c5c
Aug 12 07:11:27 dist-test-slave-csg5 krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518687, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:27.776320 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.761186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:53467 (local address 127.31.156.126:45227)
0812 07:11:27.761418 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:27.761423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:27.762672 (+ 1249us) server_negotiation.cc:407] Connection header received
0812 07:11:27.764444 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:27.764448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:27.764517 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:27.764611 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:27.766057 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.766762 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.768183 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.768451 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.768587 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.771835 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:27.771839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:27.771842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:27.771873 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:27.773662 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:27.774305 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:27.774309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:27.774310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:27.774417 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:27.774852 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:27.774856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:27.774857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:27.775097 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:27.775286 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:27.775954 (+ 668us) server_negotiation.cc:299] Negotiation successful
0812 07:11:27.776096 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260812 07:11:27.777812 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.761661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:53467)
0812 07:11:27.762411 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:27.762487 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:27.764177 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:27.764829 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:27.764837 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:27.765300 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:27.765887 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.765903 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.766923 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.766927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.768029 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.768042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.768658 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.768663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.768875 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.769586 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:27.769605 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:27.771593 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:27.773827 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:27.773834 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:27.773851 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:27.774174 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:27.774530 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:27.774533 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:27.774535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:27.774741 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:27.775318 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:27.775326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:27.775637 (+ 311us) client_negotiation.cc:769] Sending connection context
0812 07:11:27.775981 (+ 344us) client_negotiation.cc:240] Negotiation successful
0812 07:11:27.776450 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":153,"threads_started":1}
I20260812 07:11:27.779170 18351 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45227
I20260812 07:11:27.779521 18351 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:27.780081 18351 heartbeater.cc:511] Master 127.31.156.126:45227 requested a full tablet report, sending...
I20260812 07:11:27.781335 17847 ts_manager.cc:194] Registered new tserver with Master: 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863)
I20260812 07:11:27.781980 17847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:53467
I20260812 07:11:27.790959 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:11:27.803074 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.793628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59458 (local address 127.31.156.126:45227)
0812 07:11:27.793865 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:27.793869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:27.793986 (+ 117us) server_negotiation.cc:407] Connection header received
0812 07:11:27.794196 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:27.794199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:27.794250 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:27.794385 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:27.795260 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.795819 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.796721 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.796981 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.797104 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.799127 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:27.799131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:27.799133 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:27.799166 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:27.800935 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:27.801478 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:27.801482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:27.801483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:27.801581 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:27.801886 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:27.801890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:27.801892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:27.802162 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:27.802321 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:27.802733 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 07:11:27.802885 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260812 07:11:27.807273 17847 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:11:27.810891 17847 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:11:27.829356 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.821854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:35271 (local address 127.0.0.1:49754)
0812 07:11:27.822563 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:27.822626 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:27.822955 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:27.823911 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:27.823918 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:27.823946 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:27.824114 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.824122 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.825796 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.825799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.827459 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.827472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.828686 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.828691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.828841 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.828897 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 07:11:27.828998 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 07:11:27.829164 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":4,"thread_start_us":133,"threads_started":1}
I20260812 07:11:27.829543 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.823711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41718 (local address 127.31.156.67:42863)
0812 07:11:27.824961 (+ 1250us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:27.824972 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:27.824993 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:11:27.825063 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:27.825067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:27.825269 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:27.825415 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:27.825735 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.826718 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.828040 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.828838 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.828980 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.829168 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:27.829235 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 07:11:27.829346 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1143,"thread_start_us":174,"threads_started":1}
I20260812 07:11:27.829559 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.823454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:42863 (local address 127.0.0.1:41718)
0812 07:11:27.823849 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:27.823867 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:27.823964 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:27.825446 (+ 1482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:27.825450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:27.825472 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:27.825585 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.825591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.826880 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.826883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.827882 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.827894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.828981 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.828986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.829102 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.829128 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 07:11:27.829201 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 07:11:27.829302 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":3,"thread_start_us":116,"threads_started":1}
I20260812 07:11:27.830024 18364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.822069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:36725 (local address 127.0.0.1:50566)
0812 07:11:27.822563 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:27.822625 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:27.822853 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:27.824630 (+ 1777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:27.824633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:27.824652 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:27.824749 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.824754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.826633 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.826637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.827925 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.827934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.829284 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.829287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.829404 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.829434 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:11:27.829784 (+ 350us) client_negotiation.cc:240] Negotiation successful
0812 07:11:27.829867 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":115,"thread_start_us":117,"threads_started":1}
I20260812 07:11:27.830470 18366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.823178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50566 (local address 127.31.156.66:36725)
0812 07:11:27.824139 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:27.824154 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:27.824196 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:11:27.824278 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:27.824283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:27.824465 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:27.824630 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:27.824933 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.826463 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.828067 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.828934 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.829060 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.829215 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:27.830210 (+ 995us) server_negotiation.cc:299] Negotiation successful
0812 07:11:27.830324 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":153,"threads_started":1}
I20260812 07:11:27.830534 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.822074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.31.156.65:35271)
0812 07:11:27.823316 (+ 1242us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:27.823326 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:27.823358 (+ 32us) server_negotiation.cc:407] Connection header received
0812 07:11:27.823445 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:27.823451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:27.823698 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:27.824299 (+ 601us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:27.824445 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.825633 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.827611 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.828512 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.830066 (+ 1554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.830243 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:27.830295 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 07:11:27.830385 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1103,"thread_start_us":105,"threads_started":1}
I20260812 07:11:27.833356 18129 tablet_service.cc:1467] Processing CreateTablet for tablet b4090d7237d643e9b3f52c12e813f9d2 (DEFAULT_TABLE table=test-table [id=2a22aa034f6f4485a5c0fdb8f13898ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:27.833514 17983 tablet_service.cc:1467] Processing CreateTablet for tablet b4090d7237d643e9b3f52c12e813f9d2 (DEFAULT_TABLE table=test-table [id=2a22aa034f6f4485a5c0fdb8f13898ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:27.833519 18277 tablet_service.cc:1467] Processing CreateTablet for tablet b4090d7237d643e9b3f52c12e813f9d2 (DEFAULT_TABLE table=test-table [id=2a22aa034f6f4485a5c0fdb8f13898ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:27.834758 18129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4090d7237d643e9b3f52c12e813f9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:27.834785 17983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4090d7237d643e9b3f52c12e813f9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:27.834806 18277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4090d7237d643e9b3f52c12e813f9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:27.841931 18369 tablet_bootstrap.cc:492] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8: Bootstrap starting.
I20260812 07:11:27.843113 18370 tablet_bootstrap.cc:492] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81: Bootstrap starting.
I20260812 07:11:27.844525 18371 tablet_bootstrap.cc:492] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Bootstrap starting.
I20260812 07:11:27.845232 18369 tablet_bootstrap.cc:654] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:27.846264 18370 tablet_bootstrap.cc:654] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:27.846359 18369 log.cc:824] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:27.847091 18371 tablet_bootstrap.cc:654] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:27.847324 18370 log.cc:824] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:27.848055 18371 log.cc:824] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:27.848884 18369 tablet_bootstrap.cc:492] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8: No bootstrap required, opened a new log
I20260812 07:11:27.849141 18369 ts_tablet_manager.cc:1397] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:11:27.849618 18370 tablet_bootstrap.cc:492] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81: No bootstrap required, opened a new log
I20260812 07:11:27.849825 18370 ts_tablet_manager.cc:1397] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:11:27.850126 18371 tablet_bootstrap.cc:492] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: No bootstrap required, opened a new log
I20260812 07:11:27.850467 18371 ts_tablet_manager.cc:1397] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 07:11:27.853631 18369 raft_consensus.cc:348] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } }
I20260812 07:11:27.853785 18370 raft_consensus.cc:348] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } }
I20260812 07:11:27.854022 18369 raft_consensus.cc:374] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:27.854033 18370 raft_consensus.cc:374] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:27.854094 18370 raft_consensus.cc:729] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1057199d80f447ce9013ca00e6758b81, State: Initialized, Role: FOLLOWER
I20260812 07:11:27.854140 18369 raft_consensus.cc:729] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd8d839b30941f39f757b56296514f8, State: Initialized, Role: FOLLOWER
I20260812 07:11:27.854693 18369 consensus_queue.cc:260] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } }
I20260812 07:11:27.854700 18370 consensus_queue.cc:260] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } }
I20260812 07:11:27.855226 18371 raft_consensus.cc:348] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } }
I20260812 07:11:27.855531 18351 heartbeater.cc:503] Master 127.31.156.126:45227 was elected leader, sending a full tablet report...
I20260812 07:11:27.855572 18371 raft_consensus.cc:374] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:27.855634 18371 raft_consensus.cc:729] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb049cebf9644efabdec069ffd1ba8b, State: Initialized, Role: FOLLOWER
I20260812 07:11:27.855924 18369 ts_tablet_manager.cc:1428] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260812 07:11:27.856086 18371 consensus_queue.cc:260] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } }
I20260812 07:11:27.856320 18202 heartbeater.cc:503] Master 127.31.156.126:45227 was elected leader, sending a full tablet report...
I20260812 07:11:27.856834 18370 ts_tablet_manager.cc:1428] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 07:11:27.857188 18371 ts_tablet_manager.cc:1428] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260812 07:11:27.857542 18054 heartbeater.cc:503] Master 127.31.156.126:45227 was elected leader, sending a full tablet report...
I20260812 07:11:27.858556 18377 raft_consensus.cc:482] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:11:27.858826 18377 raft_consensus.cc:504] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } }
I20260812 07:11:27.860136 18377 leader_election.cc:288] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725), 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863)
I20260812 07:11:27.865914 18366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.861251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:41165 (local address 127.31.156.66:36725)
0812 07:11:27.861472 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:27.861477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:27.861626 (+ 149us) server_negotiation.cc:407] Connection header received
0812 07:11:27.861857 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:27.861861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:27.861910 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:27.861965 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:27.862451 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.863148 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.864696 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.865243 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.865354 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.865432 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:27.865664 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 07:11:27.865772 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260812 07:11:27.865962 18379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.860787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:42863 (local address 127.31.156.65:48423)
0812 07:11:27.861974 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:27.861990 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:27.862107 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:27.862421 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:27.862429 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:27.862450 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:27.862572 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.862579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.863606 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.863609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.864724 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.864736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.865475 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.865479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.865586 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.865640 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 07:11:27.865726 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 07:11:27.865799 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1118,"mutex_wait_us":4,"thread_start_us":100,"threads_started":1}
I20260812 07:11:27.865964 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.861135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:48423 (local address 127.31.156.67:42863)
0812 07:11:27.861281 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:27.861286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:27.862093 (+ 807us) server_negotiation.cc:407] Connection header received
0812 07:11:27.862202 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:27.862205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:27.862248 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:27.862328 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:27.862845 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.863459 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.864872 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.865332 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.865461 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.865514 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:27.865761 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 07:11:27.865848 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:11:27.865931 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.860348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:36725 (local address 127.31.156.65:41165)
0812 07:11:27.860981 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:27.861004 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:27.861720 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:27.862141 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:27.862143 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:27.862165 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:27.862306 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.862317 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.863276 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.863280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.864447 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:27.864457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.865401 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.865407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:27.865508 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.865537 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:11:27.865616 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 07:11:27.865676 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":105,"thread_start_us":119,"threads_started":1}
I20260812 07:11:27.867027 18298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4090d7237d643e9b3f52c12e813f9d2" candidate_uuid: "8fb049cebf9644efabdec069ffd1ba8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1057199d80f447ce9013ca00e6758b81" is_pre_election: true
I20260812 07:11:27.867017 18151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4090d7237d643e9b3f52c12e813f9d2" candidate_uuid: "8fb049cebf9644efabdec069ffd1ba8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd8d839b30941f39f757b56296514f8" is_pre_election: true
I20260812 07:11:27.867372 18151 raft_consensus.cc:2454] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fb049cebf9644efabdec069ffd1ba8b in term 0.
I20260812 07:11:27.867372 18298 raft_consensus.cc:2454] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fb049cebf9644efabdec069ffd1ba8b in term 0.
I20260812 07:11:27.867895 17931 leader_election.cc:302] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1057199d80f447ce9013ca00e6758b81, 8fb049cebf9644efabdec069ffd1ba8b; no voters:
I20260812 07:11:27.868176 18377 raft_consensus.cc:2781] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:11:27.868273 18377 raft_consensus.cc:482] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:11:27.868328 18377 raft_consensus.cc:3037] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:27.869450 18377 raft_consensus.cc:504] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } }
I20260812 07:11:27.869820 18377 leader_election.cc:288] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 1 election: Requested vote from peers ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725), 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863)
I20260812 07:11:27.870307 18151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4090d7237d643e9b3f52c12e813f9d2" candidate_uuid: "8fb049cebf9644efabdec069ffd1ba8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd8d839b30941f39f757b56296514f8"
I20260812 07:11:27.870396 18298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4090d7237d643e9b3f52c12e813f9d2" candidate_uuid: "8fb049cebf9644efabdec069ffd1ba8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1057199d80f447ce9013ca00e6758b81"
I20260812 07:11:27.870456 18151 raft_consensus.cc:3037] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:27.870515 18298 raft_consensus.cc:3037] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:27.871883 18151 raft_consensus.cc:2454] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fb049cebf9644efabdec069ffd1ba8b in term 1.
I20260812 07:11:27.871883 18298 raft_consensus.cc:2454] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fb049cebf9644efabdec069ffd1ba8b in term 1.
I20260812 07:11:27.872274 17931 leader_election.cc:302] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1057199d80f447ce9013ca00e6758b81, 8fb049cebf9644efabdec069ffd1ba8b; no voters:
I20260812 07:11:27.872835 18377 raft_consensus.cc:2781] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:11:27.873138 18377 raft_consensus.cc:686] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 1 LEADER]: Becoming Leader. State: Replica: 8fb049cebf9644efabdec069ffd1ba8b, State: Running, Role: LEADER
I20260812 07:11:27.873575 18377 consensus_queue.cc:237] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } }
I20260812 07:11:27.877473 17845 catalog_manager.cc:5495] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fb049cebf9644efabdec069ffd1ba8b (127.31.156.65). New cstate: current_term: 1 leader_uuid: "8fb049cebf9644efabdec069ffd1ba8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } health_report { overall_health: UNKNOWN } } }
I20260812 07:11:27.895292 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.891370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.31.156.65:35271)
0812 07:11:27.891538 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:27.891545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:27.891561 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:11:27.891703 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:27.891705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:27.891764 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:27.891886 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:11:27.892162 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.892754 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.893641 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:27.893869 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:27.893986 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:27.894330 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:11:27.894847 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:11:27.894983 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:27.895050 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 07:11:27.895117 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:11:27.902609 18298 raft_consensus.cc:1261] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81 [term 1 FOLLOWER]: Refusing update from remote peer 8fb049cebf9644efabdec069ffd1ba8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:27.902609 18151 raft_consensus.cc:1261] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8 [term 1 FOLLOWER]: Refusing update from remote peer 8fb049cebf9644efabdec069ffd1ba8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:27.903697 18380 consensus_queue.cc:1035] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:27.903980 18377 consensus_queue.cc:1035] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:27.925354 17845 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:11:27.925556 17845 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:11:27.928629 17845 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Sending DeleteTablet for 3 replicas of tablet b4090d7237d643e9b3f52c12e813f9d2
I20260812 07:11:27.929713 18129 tablet_service.cc:1514] Processing DeleteTablet for tablet b4090d7237d643e9b3f52c12e813f9d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:27 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:50566
I20260812 07:11:27.929697 17983 tablet_service.cc:1514] Processing DeleteTablet for tablet b4090d7237d643e9b3f52c12e813f9d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:27 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:49754
I20260812 07:11:27.929749 18277 tablet_service.cc:1514] Processing DeleteTablet for tablet b4090d7237d643e9b3f52c12e813f9d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:27 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:41718
I20260812 07:11:27.930335 18400 tablet_replica.cc:330] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81: stopping tablet replica
I20260812 07:11:27.930610 18399 tablet_replica.cc:330] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8: stopping tablet replica
I20260812 07:11:27.930612 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 17914
I20260812 07:11:27.935124 18399 raft_consensus.cc:2229] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:11:27.935568 18399 raft_consensus.cc:2258] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:27.938421 18399 ts_tablet_manager.cc:1905] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:11:27.940923 18399 ts_tablet_manager.cc:1918] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:27.941069 18399 log.cc:1196] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/b4090d7237d643e9b3f52c12e813f9d2
I20260812 07:11:27.941259 18400 raft_consensus.cc:2229] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:11:27.941473 18399 ts_tablet_manager.cc:1939] T b4090d7237d643e9b3f52c12e813f9d2 P ddd8d839b30941f39f757b56296514f8: Deleting consensus metadata
I20260812 07:11:27.941720 18400 raft_consensus.cc:2258] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:27.943045 17828 catalog_manager.cc:4841] TS ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725): tablet b4090d7237d643e9b3f52c12e813f9d2 (table test-table [id=2a22aa034f6f4485a5c0fdb8f13898ef]) successfully deleted
I20260812 07:11:27.943572 18400 ts_tablet_manager.cc:1905] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 07:11:27.944581 17830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:35271 (error 108)
I20260812 07:11:27.944828 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 18059
I20260812 07:11:27.947496 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:27.947110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:35271 (local address 127.0.0.1:49776)
0812 07:11:27.947320 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:27.947396 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:35271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260812 07:11:27.947695 17830 catalog_manager.cc:4584] TS 8fb049cebf9644efabdec069ffd1ba8b (127.31.156.65:35271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4090d7237d643e9b3f52c12e813f9d2: Network error: Client connection negotiation failed: client connection to 127.31.156.65:35271: connect: Connection refused (error 111)
I20260812 07:11:27.948082 18400 ts_tablet_manager.cc:1918] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:27.948170 18400 log.cc:1196] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/b4090d7237d643e9b3f52c12e813f9d2
I20260812 07:11:27.948433 18400 ts_tablet_manager.cc:1939] T b4090d7237d643e9b3f52c12e813f9d2 P 1057199d80f447ce9013ca00e6758b81: Deleting consensus metadata
I20260812 07:11:27.949972 17828 catalog_manager.cc:4841] TS 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863): tablet b4090d7237d643e9b3f52c12e813f9d2 (table test-table [id=2a22aa034f6f4485a5c0fdb8f13898ef]) successfully deleted
I20260812 07:11:27.955246 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 18207
I20260812 07:11:27.966048 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 17812
I20260812 07:11:47.976739 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45227
--webserver_interface=127.31.156.126
--webserver_port=33903
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:45227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:48.110526 18402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:48.111057 18402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:48.111197 18402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:48.116586 18402 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 07:11:48.116705 18402 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 07:11:48.116734 18402 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 07:11:48.116760 18402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:11:48.116808 18402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:48.116879 18402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:48.116923 18402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:11:48.116966 18402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:11:48.123390 18402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:45227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=33903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:48.124738 18402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:48.125998 18402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:48.142915 18409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:48.142879 18408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:48.142892 18411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:48.142967 18402 server_base.cc:1034] running on GCE node
I20260812 07:11:48.144066 18402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:48.145486 18402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:48.147006 18402 hybrid_clock.cc:648] HybridClock initialized: now 1786518708146964 us; error 61 us; skew 500 ppm
Aug 12 07:11:48 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518708, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:48.157145 18402 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:11:48.158847 18402 webserver.cc:460] Webserver started at http://127.31.156.126:33903/ using document root <none> and password file <none>
I20260812 07:11:48.159618 18402 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:48.159718 18402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:48.165022 18402 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:11:48.168455 18420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:48.170328 18402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:11:48.170571 18402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "3194b83909314a679fd00ec5cf818283"
format_stamp: "Formatted at 2026-08-12 07:11:26 on dist-test-slave-csg5"
server_key: "159cadc1cf25499a004c3b2db0700e31"
server_key_iv: "99843a8fbae9ce869c72025319a369c8"
server_key_version: "encryptionkey@0"
I20260812 07:11:48.171430 18402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:48.190570 18402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:48.194564 18402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:48.194854 18402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:48.203131 18402 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:45227
I20260812 07:11:48.203473 18480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:45227 every 8 connection(s)
I20260812 07:11:48.204432 18402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:11:48.207060 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 18402
I20260812 07:11:48.207543 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:35271
--local_ip_for_outbound_sockets=127.31.156.65
--tserver_master_addrs=127.31.156.126:45227
--webserver_port=42499
--webserver_interface=127.31.156.65
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:11:48.209582 18481 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:11:48.212371 18481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Bootstrap starting.
I20260812 07:11:48.239689 18481 log.cc:824] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:48.251401 18481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:11:48.251940 18481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Bootstrap complete.
I20260812 07:11:48.256585 18481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:11:48.257063 18481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3194b83909314a679fd00ec5cf818283, State: Initialized, Role: FOLLOWER
I20260812 07:11:48.257611 18481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:11:48.257779 18481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:11:48.257884 18481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:11:48.258018 18481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:11:48.260236 18481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:11:48.260617 18481 leader_election.cc:302] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3194b83909314a679fd00ec5cf818283; no voters:
I20260812 07:11:48.260989 18481 leader_election.cc:288] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:11:48.261170 18486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:11:48.261436 18486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 2 LEADER]: Becoming Leader. State: Replica: 3194b83909314a679fd00ec5cf818283, State: Running, Role: LEADER
I20260812 07:11:48.261854 18486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:11:48.262346 18481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:11:48.263547 18487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3194b83909314a679fd00ec5cf818283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } } }
I20260812 07:11:48.263705 18487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:48.263993 18488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3194b83909314a679fd00ec5cf818283. Latest consensus state: current_term: 2 leader_uuid: "3194b83909314a679fd00ec5cf818283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } } }
I20260812 07:11:48.264077 18488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:48.264112 18495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:11:48.269520 18495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:11:48.270267 18495 catalog_manager.cc:1259] Loaded cluster ID: 0c991bcf85f749eb94f523b2235cc174
I20260812 07:11:48.270498 18495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:11:48.273416 18495 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:11:48.274222 18495 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Loaded TSK: 0
I20260812 07:11:48.289297 18495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Generated new TSK 1
I20260812 07:11:48.289623 18495 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:11:48.364523 18483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:48.364853 18483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:48.364934 18483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:48.369648 18483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:48.369742 18483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:48.369807 18483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:11:48.376022 18483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:35271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=42499
--tserver_master_addrs=127.31.156.126:45227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:48.377626 18483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:48.378862 18483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:48.391566 18512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:48.391561 18515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:48.391887 18483 server_base.cc:1034] running on GCE node
W20260812 07:11:48.391577 18513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:48.392709 18483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:48.393709 18483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:48.394894 18483 hybrid_clock.cc:648] HybridClock initialized: now 1786518708394869 us; error 64 us; skew 500 ppm
Aug 12 07:11:48 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518708, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:48.398057 18483 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:11:48.399233 18483 webserver.cc:460] Webserver started at http://127.31.156.65:42499/ using document root <none> and password file <none>
I20260812 07:11:48.399844 18483 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:48.399930 18483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:48.404299 18483 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:11:48.406893 18523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:48.408341 18483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:48.408453 18483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "8fb049cebf9644efabdec069ffd1ba8b"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "6f14d200dcc99d717f6083f976cf49e1"
server_key_iv: "d1e8bed248a2c2026d1fba4a7ccea77b"
server_key_version: "encryptionkey@0"
I20260812 07:11:48.409147 18483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:48.433676 18483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:48.437894 18483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:48.438159 18483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:48.439009 18483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:48.440369 18532 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 07:11:48.445030 18483 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 07:11:48.445150 18483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 07:11:48.445277 18483 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 07:11:48.447358 18483 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 07:11:48.447414 18483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260812 07:11:48.447755 18532 tablet_bootstrap.cc:492] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Bootstrap starting.
I20260812 07:11:48.459411 18483 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:35271
I20260812 07:11:48.459447 18652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:35271 every 8 connection(s)
I20260812 07:11:48.460728 18483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:11:48.469316 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 18483
I20260812 07:11:48.469859 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:36725
--local_ip_for_outbound_sockets=127.31.156.66
--tserver_master_addrs=127.31.156.126:45227
--webserver_port=38261
--webserver_interface=127.31.156.66
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:11:48.470780 18532 log.cc:824] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:11:48 dist-test-slave-csg5 krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518708, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:48.480455 18532 tablet_bootstrap.cc:492] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:11:48.481098 18532 tablet_bootstrap.cc:492] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Bootstrap complete.
I20260812 07:11:48.482745 18532 ts_tablet_manager.cc:1397] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Time spent bootstrapping tablet: real 0.035s user 0.026s sys 0.008s
I20260812 07:11:48.485173 18659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.463588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:36069)
0812 07:11:48.464389 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:48.464471 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:48.465788 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:48.467353 (+ 1565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:48.467364 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:48.468099 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:48.468946 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.468973 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.471173 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.471177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.472662 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.472682 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.473763 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.473771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.474029 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.474903 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:48.474934 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:48.477252 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:48.480829 (+ 3577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:48.480839 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:48.480858 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:48.481293 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:48.481839 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:48.481846 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:48.481852 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:48.482067 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:48.483026 (+ 959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:48.483035 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:48.483370 (+ 335us) client_negotiation.cc:769] Sending connection context
0812 07:11:48.483606 (+ 236us) client_negotiation.cc:240] Negotiation successful
0812 07:11:48.484017 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":188,"threads_started":1}
I20260812 07:11:48.485432 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.463528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:36069 (local address 127.31.156.126:45227)
0812 07:11:48.463988 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:48.464009 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:48.464658 (+ 649us) server_negotiation.cc:407] Connection header received
0812 07:11:48.466357 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:48.466369 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:48.466844 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:48.467173 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:48.469239 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.470928 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.472847 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.473570 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.473788 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.477728 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:48.477749 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:48.477772 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:48.477824 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:48.480635 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.481465 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.481476 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.481483 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.481688 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.482189 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.482196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.482199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.482859 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:48.483022 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:48.483793 (+ 771us) server_negotiation.cc:299] Negotiation successful
0812 07:11:48.484196 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":141,"threads_started":1}
I20260812 07:11:48.486311 18532 raft_consensus.cc:348] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } }
I20260812 07:11:48.486902 18532 raft_consensus.cc:729] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb049cebf9644efabdec069ffd1ba8b, State: Initialized, Role: FOLLOWER
I20260812 07:11:48.487494 18532 consensus_queue.cc:260] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } }
I20260812 07:11:48.488631 18654 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45227
I20260812 07:11:48.488627 18532 ts_tablet_manager.cc:1428] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 07:11:48.488947 18654 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:48.489566 18654 heartbeater.cc:511] Master 127.31.156.126:45227 requested a full tablet report, sending...
I20260812 07:11:48.492779 18436 ts_manager.cc:194] Registered new tserver with Master: 8fb049cebf9644efabdec069ffd1ba8b (127.31.156.65:35271)
I20260812 07:11:48.497522 18436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:36069
I20260812 07:11:48.500459 18654 heartbeater.cc:503] Master 127.31.156.126:45227 was elected leader, sending a full tablet report...
Aug 12 07:11:48 dist-test-slave-csg5 krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518708, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.65@KRBTEST.COM
I20260812 07:11:48.509440 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.496708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:35271 (local address 127.0.0.1:56942)
0812 07:11:48.497586 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:48.497642 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:48.497965 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:48.498506 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:48.498512 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:48.499048 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:48.499192 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.499202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.500705 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.500708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.501675 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.501689 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.502530 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.502534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.502728 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.503366 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:48.503379 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:48.505088 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:48.507350 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:48.507362 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:48.507378 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:48.507651 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:48.508019 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:48.508023 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:48.508025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:48.508132 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:48.508691 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:48.508698 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:48.508837 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 07:11:48.508990 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 07:11:48.509241 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":146,"threads_started":1}
I20260812 07:11:48.509440 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.496827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56942 (local address 127.31.156.65:35271)
0812 07:11:48.497286 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:48.497300 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:48.497845 (+ 545us) server_negotiation.cc:407] Connection header received
0812 07:11:48.498146 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:48.498151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:48.498345 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:48.498505 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:48.499354 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.500528 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.501845 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.502400 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.502523 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.505280 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:48.505294 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:48.505309 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:48.505340 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:48.507200 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.507811 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.507816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.507817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.507910 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.508243 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.508246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.508248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.508553 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:48.508723 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:48.509079 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 07:11:48.509253 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":123,"threads_started":1}
I20260812 07:11:48.510550 18580 tablet_service.cc:1514] Processing DeleteTablet for tablet b4090d7237d643e9b3f52c12e813f9d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:27 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:56942
I20260812 07:11:48.511584 18665 tablet_replica.cc:330] stopping tablet replica
I20260812 07:11:48.511960 18665 raft_consensus.cc:2229] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:11:48.512259 18665 raft_consensus.cc:2258] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:48.513510 18665 ts_tablet_manager.cc:1905] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:11:48.518638 18665 ts_tablet_manager.cc:1918] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:48.518823 18665 log.cc:1196] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal/wals/b4090d7237d643e9b3f52c12e813f9d2
I20260812 07:11:48.519126 18665 ts_tablet_manager.cc:1939] T b4090d7237d643e9b3f52c12e813f9d2 P 8fb049cebf9644efabdec069ffd1ba8b: Deleting consensus metadata
I20260812 07:11:48.521754 18423 catalog_manager.cc:4841] TS 8fb049cebf9644efabdec069ffd1ba8b (127.31.156.65:35271): tablet b4090d7237d643e9b3f52c12e813f9d2 (table test-table [id=2a22aa034f6f4485a5c0fdb8f13898ef]) successfully deleted
W20260812 07:11:48.609383 18661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:48.609741 18661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:48.609794 18661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:48.615103 18661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:48.615202 18661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:48.615311 18661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:11:48.621739 18661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:36725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=38261
--tserver_master_addrs=127.31.156.126:45227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:48.623282 18661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:48.624536 18661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:48.636286 18672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:48.636363 18674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:48.636415 18671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:48.637254 18661 server_base.cc:1034] running on GCE node
I20260812 07:11:48.637876 18661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:48.638643 18661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:48.640069 18661 hybrid_clock.cc:648] HybridClock initialized: now 1786518708640016 us; error 77 us; skew 500 ppm
Aug 12 07:11:48 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518708, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:48.643816 18661 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:11:48.645009 18661 webserver.cc:460] Webserver started at http://127.31.156.66:38261/ using document root <none> and password file <none>
I20260812 07:11:48.645740 18661 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:48.645882 18661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:48.650516 18661 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 07:11:48.653311 18681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:48.654863 18661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:48.654984 18661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "ddd8d839b30941f39f757b56296514f8"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "a776cab3e01594e2ca18a31c5d71dbe1"
server_key_iv: "0e29f6cd1cc666aa525182bd1e5d1e06"
server_key_version: "encryptionkey@0"
I20260812 07:11:48.655594 18661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:48.684336 18661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:48.688161 18661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:48.688446 18661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:48.689060 18661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:48.689976 18661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:48.690026 18661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:48.690068 18661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:48.690085 18661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:48.699942 18661 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:36725
I20260812 07:11:48.700002 18812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:36725 every 8 connection(s)
I20260812 07:11:48.701133 18661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:11:48.711015 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 18661
I20260812 07:11:48.711530 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:42863
--local_ip_for_outbound_sockets=127.31.156.67
--tserver_master_addrs=127.31.156.126:45227
--webserver_port=43561
--webserver_interface=127.31.156.67
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:11:48 dist-test-slave-csg5 krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518708, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:48.717988 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.703488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:34195 (local address 127.31.156.126:45227)
0812 07:11:48.703688 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:48.703692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:48.704695 (+ 1003us) server_negotiation.cc:407] Connection header received
0812 07:11:48.706293 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:48.706297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:48.706374 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:48.706487 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:48.707979 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.708564 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.709864 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.710123 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.710262 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.713664 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:48.713668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:48.713670 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:48.713701 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:48.715466 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.716159 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.716164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.716168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.716264 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.716670 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.716673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.716675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.716900 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:48.717021 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:48.717610 (+ 589us) server_negotiation.cc:299] Negotiation successful
0812 07:11:48.717797 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 07:11:48.718998 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.703929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:34195)
0812 07:11:48.704500 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:48.704552 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:48.706001 (+ 1449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:48.706786 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:48.706795 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:48.707235 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:48.707782 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.707799 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.708730 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.708739 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.709693 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.709703 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.710319 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.710323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.710520 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.711377 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:48.711404 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:48.713417 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:48.715628 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:48.715636 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:48.715658 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:48.716033 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:48.716385 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:48.716388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:48.716390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:48.716564 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:48.717020 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:48.717030 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:48.717339 (+ 309us) client_negotiation.cc:769] Sending connection context
0812 07:11:48.717556 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 07:11:48.717970 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":135,"threads_started":1}
I20260812 07:11:48.720419 18813 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45227
I20260812 07:11:48.720723 18813 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:48.721307 18813 heartbeater.cc:511] Master 127.31.156.126:45227 requested a full tablet report, sending...
I20260812 07:11:48.722582 18436 ts_manager.cc:194] Registered new tserver with Master: ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725)
I20260812 07:11:48.723570 18436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:34195
W20260812 07:11:48.842705 18817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:48.843072 18817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:48.843145 18817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:48.847870 18817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:48.847952 18817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:48.848054 18817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:11:48.854065 18817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:42863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=43561
--tserver_master_addrs=127.31.156.126:45227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:11:48.855409 18817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:48.856581 18817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:48.867976 18823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:48.867987 18824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:48.868005 18826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:48.868047 18817 server_base.cc:1034] running on GCE node
I20260812 07:11:48.868824 18817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:48.869581 18817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:48.870747 18817 hybrid_clock.cc:648] HybridClock initialized: now 1786518708870727 us; error 30 us; skew 500 ppm
Aug 12 07:11:48 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518708, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:48.873958 18817 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:11:48.874998 18817 webserver.cc:460] Webserver started at http://127.31.156.67:43561/ using document root <none> and password file <none>
I20260812 07:11:48.875665 18817 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:48.875756 18817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:48.880142 18817 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:11:48.882534 18834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:48.884478 18817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:48.884620 18817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "1057199d80f447ce9013ca00e6758b81"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "dde7b80e2d3741d7fceca89a7fa41676"
server_key_iv: "bdcd00831e56605d752eb8f9d9e66d79"
server_key_version: "encryptionkey@0"
I20260812 07:11:48.885344 18817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:48.902177 18817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:48.906322 18817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:48.906623 18817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:48.907392 18817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:48.908453 18817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:48.908530 18817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:48.908609 18817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:48.908653 18817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:48.918924 18817 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:42863
I20260812 07:11:48.918944 18957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:42863 every 8 connection(s)
I20260812 07:11:48.920132 18817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:11:48.921038 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 18817
I20260812 07:11:48.932399 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.921988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46676 (local address 127.31.156.126:45227)
0812 07:11:48.922193 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:48.922199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:48.922459 (+ 260us) server_negotiation.cc:407] Connection header received
0812 07:11:48.922686 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:48.922690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:48.922750 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:48.922848 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:48.923786 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.924421 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.926065 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.926273 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.926428 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.928363 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:48.928367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:48.928370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:48.928400 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:48.930147 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.930845 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.930849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.930851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.930945 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.931400 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.931402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.931403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.931585 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:48.931742 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:48.932057 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 07:11:48.932218 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 07:11:48 dist-test-slave-csg5 krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518708, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:11:48.939174 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.922585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:38845 (local address 127.31.156.126:45227)
0812 07:11:48.922887 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:48.922891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:48.923995 (+ 1104us) server_negotiation.cc:407] Connection header received
0812 07:11:48.925852 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:48.925855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:48.925904 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:48.925985 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:48.928042 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.928880 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.930734 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.931138 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.931245 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.934845 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:48.934848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:48.934850 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:48.934875 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:48.936458 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.937045 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.937049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.937050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.937149 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.937746 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.937750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.937751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.937970 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:48.938060 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:48.938778 (+ 718us) server_negotiation.cc:299] Negotiation successful
0812 07:11:48.938984 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":107,"threads_started":1}
I20260812 07:11:48.940032 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.923231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:38845)
0812 07:11:48.923846 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:48.923895 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:48.925534 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:48.926180 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:48.926188 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:48.926988 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:48.927845 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.927874 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.929104 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.929110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.930514 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.930528 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.931330 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.931339 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.931556 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.932343 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:48.932367 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:48.934574 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:48.936588 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:48.936593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:48.936609 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:48.936928 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:48.937279 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:48.937282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:48.937283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:48.937451 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:48.938097 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:48.938103 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:48.938388 (+ 285us) client_negotiation.cc:769] Sending connection context
0812 07:11:48.938577 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 07:11:48.939067 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":116,"threads_started":1}
I20260812 07:11:48.941484 18958 heartbeater.cc:347] Connected to a master server at 127.31.156.126:45227
I20260812 07:11:48.941789 18958 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:48.942296 18958 heartbeater.cc:511] Master 127.31.156.126:45227 requested a full tablet report, sending...
I20260812 07:11:48.943480 18436 ts_manager.cc:194] Registered new tserver with Master: 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863)
I20260812 07:11:48.944046 18436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:38845
I20260812 07:11:48.947389 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:11:48.952891 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.948139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46686 (local address 127.31.156.126:45227)
0812 07:11:48.948293 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:48.948297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:48.948593 (+ 296us) server_negotiation.cc:407] Connection header received
0812 07:11:48.948832 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:48.948835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:48.948943 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:48.949047 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:11:48.949420 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.949956 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.951390 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.951690 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.951797 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.952196 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:11:48.952463 (+ 267us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 07:11:48.952753 (+ 290us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46686: token expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 07:11:48.953011 18961 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46686: token expired
W20260812 07:11:48.953436 18356 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.156.126:45227)
I20260812 07:11:48.963128 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.953918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46692 (local address 127.31.156.126:45227)
0812 07:11:48.954158 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:48.954162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:48.954175 (+ 13us) server_negotiation.cc:407] Connection header received
0812 07:11:48.954210 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:48.954212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:48.954254 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:48.954345 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:48.955199 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.955829 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.956785 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.957028 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.957129 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.959002 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:48.959005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:48.959007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:48.959034 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:48.960823 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.961372 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.961376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.961377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.961475 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:48.961882 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:48.961886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:48.961887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:48.962209 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:48.962369 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:48.962856 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 07:11:48.962985 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260812 07:11:48.966413 18436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:11:48.970116 18436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:11:48.979801 18580 tablet_service.cc:1467] Processing CreateTablet for tablet 8c7ccc976b51436e93637480ec71f88f (DEFAULT_TABLE table=test-table [id=ba4ab317ea9b44db847244c7b9b6433e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:48.980623 18580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c7ccc976b51436e93637480ec71f88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:48.983938 18532 tablet_bootstrap.cc:492] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b: Bootstrap starting.
I20260812 07:11:48.985086 18532 tablet_bootstrap.cc:654] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:48.985376 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.978767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:42863 (local address 127.0.0.1:53588)
0812 07:11:48.978919 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:48.978940 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:48.979035 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:48.980439 (+ 1404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:48.980442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:48.980463 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:48.980594 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.980602 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.982503 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.982507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.983917 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.983928 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.984884 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.984888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.984983 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.985032 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 07:11:48.985138 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 07:11:48.985231 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":38}
I20260812 07:11:48.985427 18970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.978956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53588 (local address 127.31.156.67:42863)
0812 07:11:48.979414 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:48.979429 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:48.979461 (+ 32us) server_negotiation.cc:407] Connection header received
0812 07:11:48.979560 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:48.979567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:48.980234 (+ 667us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:48.980881 (+ 647us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:48.981076 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.982347 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.984054 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.984745 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.984884 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.985055 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:48.985182 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 07:11:48.985284 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":202,"threads_started":1}
I20260812 07:11:48.986969 18972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.979724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.31.156.66:36725)
0812 07:11:48.980217 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:48.980230 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:48.980268 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:11:48.980367 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:48.980376 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:48.980578 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:48.980758 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:48.980968 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.982363 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.984502 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.985987 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.986436 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.986600 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:48.986687 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 07:11:48.986805 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":146,"threads_started":1}
I20260812 07:11:48.987703 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:48.979081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:36725 (local address 127.0.0.1:41238)
0812 07:11:48.979467 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:48.979482 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:48.979578 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:48.980740 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:48.980743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:48.980763 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:48.980856 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.980862 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.982548 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.982552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.984334 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:48.984346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:48.986142 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:48.986146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:48.986236 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:48.986260 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:11:48.987433 (+ 1173us) client_negotiation.cc:240] Negotiation successful
0812 07:11:48.987549 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":325,"thread_start_us":143,"threads_started":1}
I20260812 07:11:48.988930 18887 tablet_service.cc:1467] Processing CreateTablet for tablet 8c7ccc976b51436e93637480ec71f88f (DEFAULT_TABLE table=test-table [id=ba4ab317ea9b44db847244c7b9b6433e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:48.990217 18887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c7ccc976b51436e93637480ec71f88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:48.990852 18532 tablet_bootstrap.cc:492] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b: No bootstrap required, opened a new log
I20260812 07:11:48.990986 18532 ts_tablet_manager.cc:1397] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260812 07:11:48.991533 18532 raft_consensus.cc:348] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:11:48.991677 18532 raft_consensus.cc:374] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:48.991730 18532 raft_consensus.cc:729] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb049cebf9644efabdec069ffd1ba8b, State: Initialized, Role: FOLLOWER
I20260812 07:11:48.991895 18532 consensus_queue.cc:260] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:11:48.992316 18532 ts_tablet_manager.cc:1428] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 07:11:48.992524 18739 tablet_service.cc:1467] Processing CreateTablet for tablet 8c7ccc976b51436e93637480ec71f88f (DEFAULT_TABLE table=test-table [id=ba4ab317ea9b44db847244c7b9b6433e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:48.993772 18739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c7ccc976b51436e93637480ec71f88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:49.000372 18976 tablet_bootstrap.cc:492] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81: Bootstrap starting.
I20260812 07:11:49.002774 18977 tablet_bootstrap.cc:492] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8: Bootstrap starting.
I20260812 07:11:49.003054 18976 tablet_bootstrap.cc:654] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:49.003957 18976 log.cc:824] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:49.005609 18977 tablet_bootstrap.cc:654] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:49.006408 18976 tablet_bootstrap.cc:492] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81: No bootstrap required, opened a new log
I20260812 07:11:49.006603 18977 log.cc:824] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:49.006722 18976 ts_tablet_manager.cc:1397] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:11:49.009004 18977 tablet_bootstrap.cc:492] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8: No bootstrap required, opened a new log
I20260812 07:11:49.009392 18977 ts_tablet_manager.cc:1397] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 07:11:49.010449 18976 raft_consensus.cc:348] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:11:49.010834 18976 raft_consensus.cc:374] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:49.010963 18976 raft_consensus.cc:729] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1057199d80f447ce9013ca00e6758b81, State: Initialized, Role: FOLLOWER
I20260812 07:11:49.011615 18976 consensus_queue.cc:260] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:11:49.012446 18958 heartbeater.cc:503] Master 127.31.156.126:45227 was elected leader, sending a full tablet report...
I20260812 07:11:49.012903 18976 ts_tablet_manager.cc:1428] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 07:11:49.013432 18977 raft_consensus.cc:348] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:11:49.013834 18977 raft_consensus.cc:374] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:49.013897 18977 raft_consensus.cc:729] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd8d839b30941f39f757b56296514f8, State: Initialized, Role: FOLLOWER
I20260812 07:11:49.014413 18977 consensus_queue.cc:260] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:11:49.015725 18977 ts_tablet_manager.cc:1428] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260812 07:11:49.016155 18813 heartbeater.cc:503] Master 127.31.156.126:45227 was elected leader, sending a full tablet report...
I20260812 07:11:49.022068 18975 raft_consensus.cc:482] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:11:49.022377 18975 raft_consensus.cc:504] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:11:49.023939 18975 leader_election.cc:288] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863), ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725)
I20260812 07:11:49.030961 18972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:49.024725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:42783 (local address 127.31.156.66:36725)
0812 07:11:49.024856 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:49.024859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:49.024873 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:11:49.024987 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:49.024990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:49.025043 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:49.025144 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:49.026360 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:49.027308 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:49.029379 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:49.030110 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:49.030351 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:49.030419 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:49.030710 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 07:11:49.030814 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:11:49.030961 18970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:49.024289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:49025 (local address 127.31.156.67:42863)
0812 07:11:49.024432 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:49.024436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:49.024740 (+ 304us) server_negotiation.cc:407] Connection header received
0812 07:11:49.024996 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:49.024999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:49.025045 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:49.025143 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:49.026359 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:49.027312 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:49.029379 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:49.030110 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:49.030262 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:49.030326 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:49.030710 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 07:11:49.030818 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:11:49.031045 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:49.024192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:42863 (local address 127.31.156.65:49025)
0812 07:11:49.024616 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:49.024634 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:49.024847 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:49.025828 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:49.025831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:49.025881 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:49.026165 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:49.026173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:49.027488 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:49.027492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:49.029195 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:49.029211 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:49.030282 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:49.030287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:49.030397 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:49.030540 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 07:11:49.030631 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 07:11:49.030797 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":270,"spinlock_wait_cycles":33792,"thread_start_us":99,"threads_started":1}
I20260812 07:11:49.031711 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:49.024535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:36725 (local address 127.31.156.65:42783)
0812 07:11:49.024794 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:49.024807 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:49.024874 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:49.025828 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:49.025831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:49.025883 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:49.026165 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:49.026173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:49.027488 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:49.027492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:49.029195 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:49.029211 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:49.030340 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:49.030343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:49.030445 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:49.030536 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 07:11:49.030632 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 07:11:49.030797 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":250,"spinlock_wait_cycles":10880,"thread_start_us":106,"threads_started":1}
I20260812 07:11:49.031811 18908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c7ccc976b51436e93637480ec71f88f" candidate_uuid: "8fb049cebf9644efabdec069ffd1ba8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1057199d80f447ce9013ca00e6758b81" is_pre_election: true
I20260812 07:11:49.032217 18908 raft_consensus.cc:2454] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fb049cebf9644efabdec069ffd1ba8b in term 0.
I20260812 07:11:49.032788 18524 leader_election.cc:302] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1057199d80f447ce9013ca00e6758b81, 8fb049cebf9644efabdec069ffd1ba8b; no voters:
I20260812 07:11:49.032868 18761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c7ccc976b51436e93637480ec71f88f" candidate_uuid: "8fb049cebf9644efabdec069ffd1ba8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd8d839b30941f39f757b56296514f8" is_pre_election: true
I20260812 07:11:49.033239 18761 raft_consensus.cc:2454] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fb049cebf9644efabdec069ffd1ba8b in term 0.
I20260812 07:11:49.033115 18975 raft_consensus.cc:2781] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:11:49.033344 18975 raft_consensus.cc:482] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:11:49.033406 18975 raft_consensus.cc:3037] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:49.034813 18975 raft_consensus.cc:504] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:11:49.035260 18975 leader_election.cc:288] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 1 election: Requested vote from peers 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863), ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725)
I20260812 07:11:49.035686 18908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c7ccc976b51436e93637480ec71f88f" candidate_uuid: "8fb049cebf9644efabdec069ffd1ba8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1057199d80f447ce9013ca00e6758b81"
I20260812 07:11:49.035758 18761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c7ccc976b51436e93637480ec71f88f" candidate_uuid: "8fb049cebf9644efabdec069ffd1ba8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd8d839b30941f39f757b56296514f8"
I20260812 07:11:49.035916 18908 raft_consensus.cc:3037] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:49.035928 18761 raft_consensus.cc:3037] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:49.037142 18761 raft_consensus.cc:2454] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fb049cebf9644efabdec069ffd1ba8b in term 1.
I20260812 07:11:49.037182 18908 raft_consensus.cc:2454] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fb049cebf9644efabdec069ffd1ba8b in term 1.
I20260812 07:11:49.037549 18524 leader_election.cc:302] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fb049cebf9644efabdec069ffd1ba8b, ddd8d839b30941f39f757b56296514f8; no voters:
I20260812 07:11:49.037762 18975 raft_consensus.cc:2781] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:11:49.038014 18975 raft_consensus.cc:686] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 LEADER]: Becoming Leader. State: Replica: 8fb049cebf9644efabdec069ffd1ba8b, State: Running, Role: LEADER
I20260812 07:11:49.038375 18975 consensus_queue.cc:237] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:11:49.041944 18436 catalog_manager.cc:5495] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fb049cebf9644efabdec069ffd1ba8b (127.31.156.65). New cstate: current_term: 1 leader_uuid: "8fb049cebf9644efabdec069ffd1ba8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } health_report { overall_health: HEALTHY } } }
I20260812 07:11:49.056748 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:49.052151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56956 (local address 127.31.156.65:35271)
0812 07:11:49.052485 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:49.052492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:49.052506 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:11:49.052588 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:49.052590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:49.052643 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:49.052757 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:11:49.053050 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:49.053758 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:49.054825 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:49.055091 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:49.055197 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:49.055607 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:11:49.056176 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:11:49.056284 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:49.056516 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 07:11:49.056585 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":159,"threads_started":1}
I20260812 07:11:49.061656 18908 raft_consensus.cc:1261] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81 [term 1 FOLLOWER]: Refusing update from remote peer 8fb049cebf9644efabdec069ffd1ba8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:49.061662 18761 raft_consensus.cc:1261] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8 [term 1 FOLLOWER]: Refusing update from remote peer 8fb049cebf9644efabdec069ffd1ba8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:49.062624 18984 consensus_queue.cc:1035] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:49.063135 18975 consensus_queue.cc:1035] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:49.073263 18989 mvcc.cc:205] Tried to move back new op lower bound from 7317580632308879360 to 7317580632224526336. Current Snapshot: MvccSnapshot[applied={T|T < 7317580632308879360}]
I20260812 07:11:49.080791 18438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:11:49.081033 18438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:11:49.084149 18438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Sending DeleteTablet for 3 replicas of tablet 8c7ccc976b51436e93637480ec71f88f
I20260812 07:11:49.085263 18887 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c7ccc976b51436e93637480ec71f88f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:49 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:53588
I20260812 07:11:49.085549 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 18483
I20260812 07:11:49.085510 18739 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c7ccc976b51436e93637480ec71f88f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:49 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:41238
I20260812 07:11:49.088868 18998 tablet_replica.cc:330] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81: stopping tablet replica
I20260812 07:11:49.089354 18998 raft_consensus.cc:2229] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:11:49.089802 18998 raft_consensus.cc:2258] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:49.092689 18998 ts_tablet_manager.cc:1905] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:11:49.096240 18998 ts_tablet_manager.cc:1918] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:49.096361 18998 log.cc:1196] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/8c7ccc976b51436e93637480ec71f88f
I20260812 07:11:49.096727 18998 ts_tablet_manager.cc:1939] T 8c7ccc976b51436e93637480ec71f88f P 1057199d80f447ce9013ca00e6758b81: Deleting consensus metadata
I20260812 07:11:49.096722 18999 tablet_replica.cc:330] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8: stopping tablet replica
I20260812 07:11:49.097195 18999 raft_consensus.cc:2229] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:11:49.097618 18999 raft_consensus.cc:2258] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:49.098516 18421 catalog_manager.cc:4841] TS 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863): tablet 8c7ccc976b51436e93637480ec71f88f (table test-table [id=ba4ab317ea9b44db847244c7b9b6433e]) successfully deleted
I20260812 07:11:49.099674 18999 ts_tablet_manager.cc:1905] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 07:11:49.103670 18423 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:35271 (error 108)
I20260812 07:11:49.103746 18999 ts_tablet_manager.cc:1918] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:49.103852 18999 log.cc:1196] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/8c7ccc976b51436e93637480ec71f88f
I20260812 07:11:49.104286 18999 ts_tablet_manager.cc:1939] T 8c7ccc976b51436e93637480ec71f88f P ddd8d839b30941f39f757b56296514f8: Deleting consensus metadata
I20260812 07:11:49.104809 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 18661
I20260812 07:11:49.105172 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:49.104777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:35271 (local address 127.0.0.1:56960)
0812 07:11:49.104962 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:49.105044 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:35271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260812 07:11:49.107084 18423 catalog_manager.cc:4584] TS 8fb049cebf9644efabdec069ffd1ba8b (127.31.156.65:35271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c7ccc976b51436e93637480ec71f88f: Network error: Client connection negotiation failed: client connection to 127.31.156.65:35271: connect: Connection refused (error 111)
I20260812 07:11:49.114781 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 18817
I20260812 07:11:49.119628 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:49.119169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:36725 (local address 127.0.0.1:41248)
0812 07:11:49.119387 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:49.119482 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.66:36725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 07:11:49.125458 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 18402
I20260812 07:12:09.138468 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45227
--webserver_interface=127.31.156.126
--webserver_port=33903
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:45227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:09.262701 19001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:09.263078 19001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:09.263164 19001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:09.267442 19001 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 07:12:09.267545 19001 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 07:12:09.267572 19001 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 07:12:09.267596 19001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:09.267639 19001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:09.267684 19001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:09.267743 19001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:09.267786 19001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:09.273851 19001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:45227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:45227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=33903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:09.275157 19001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:09.276352 19001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:12:09.287122 19001 server_base.cc:1034] running on GCE node
W20260812 07:12:09.287040 19011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:09.287042 19008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:09.287142 19009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:09.288158 19001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:09.289547 19001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:09.290779 19001 hybrid_clock.cc:648] HybridClock initialized: now 1786518729290748 us; error 47 us; skew 500 ppm
Aug 12 07:12:09 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518729, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:09.299156 19001 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:12:09.300422 19001 webserver.cc:460] Webserver started at http://127.31.156.126:33903/ using document root <none> and password file <none>
I20260812 07:12:09.301115 19001 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:09.301234 19001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:09.306314 19001 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:12:09.309451 19019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:09.311108 19001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:09.311285 19001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "3194b83909314a679fd00ec5cf818283"
format_stamp: "Formatted at 2026-08-12 07:11:26 on dist-test-slave-csg5"
server_key: "159cadc1cf25499a004c3b2db0700e31"
server_key_iv: "99843a8fbae9ce869c72025319a369c8"
server_key_version: "encryptionkey@0"
I20260812 07:12:09.312021 19001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:09.349402 19001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:09.353833 19001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:09.354122 19001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:09.363139 19001 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:45227
I20260812 07:12:09.363654 19075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:45227 every 8 connection(s)
I20260812 07:12:09.364588 19001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:09.368671 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 19001
I20260812 07:12:09.369225 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:35271
--local_ip_for_outbound_sockets=127.31.156.65
--tserver_master_addrs=127.31.156.126:45227
--webserver_port=42499
--webserver_interface=127.31.156.65
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:12:09.369997 19076 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:12:09.372049 19076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Bootstrap starting.
I20260812 07:12:09.392604 19076 log.cc:824] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:09.404698 19076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:12:09.405179 19076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Bootstrap complete.
I20260812 07:12:09.410005 19076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:12:09.410516 19076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3194b83909314a679fd00ec5cf818283, State: Initialized, Role: FOLLOWER
I20260812 07:12:09.411175 19076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:12:09.411339 19076 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:09.411438 19076 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:09.411545 19076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 2 FOLLOWER]: Advancing to term 3
I20260812 07:12:09.413713 19076 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:12:09.414096 19076 leader_election.cc:302] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3194b83909314a679fd00ec5cf818283; no voters:
I20260812 07:12:09.414513 19076 leader_election.cc:288] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 07:12:09.414602 19080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 07:12:09.414856 19080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [term 3 LEADER]: Becoming Leader. State: Replica: 3194b83909314a679fd00ec5cf818283, State: Running, Role: LEADER
I20260812 07:12:09.415221 19080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } }
I20260812 07:12:09.415925 19076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:09.416339 19082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3194b83909314a679fd00ec5cf818283. Latest consensus state: current_term: 3 leader_uuid: "3194b83909314a679fd00ec5cf818283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } } }
I20260812 07:12:09.416473 19082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:09.416759 19081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3194b83909314a679fd00ec5cf818283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3194b83909314a679fd00ec5cf818283" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 45227 } } }
I20260812 07:12:09.416884 19081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:09.417614 19085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:09.423308 19085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:09.424075 19085 catalog_manager.cc:1259] Loaded cluster ID: 0c991bcf85f749eb94f523b2235cc174
I20260812 07:12:09.424149 19085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:09.426892 19085 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:09.427824 19085 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Loaded TSK: 0
I20260812 07:12:09.427934 19085 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3194b83909314a679fd00ec5cf818283: Loaded TSK: 1
F20260812 07:12:09.428843 19085 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff16fd419ad google::LogMessage::Fail() at ??:0
@ 0x7ff16fd4488a google::LogMessage::SendToLog() at ??:0
@ 0x7ff16fd414de google::LogMessage::Flush() at ??:0
@ 0x7ff16fd42a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff172a0518d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff1729fdf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff17a6a67bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff17a68099c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff17a681178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7ff17a6c2df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ff17a6e182b std::function<>::operator()() at ??:0
@ 0x7ff17a680b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ff17a681f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff17a67e579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ff17a6c2640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff171f933d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff171f93c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ff171f953f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff171f84512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff1735c86db start_thread at ??:0
@ 0x7ff16e83f61f clone at ??:0
W20260812 07:12:09.544801 19078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:09.545125 19078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:09.545205 19078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:09.549786 19078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:09.549872 19078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:09.549937 19078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:12:09.555922 19078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:35271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=42499
--tserver_master_addrs=127.31.156.126:45227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:09.557219 19078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:09.558390 19078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:09.569898 19129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:09.569872 19128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:09.569872 19131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:09.570502 19078 server_base.cc:1034] running on GCE node
I20260812 07:12:09.571138 19078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:09.572086 19078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:09.573333 19078 hybrid_clock.cc:648] HybridClock initialized: now 1786518729573303 us; error 41 us; skew 500 ppm
Aug 12 07:12:09 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518729, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:09.577976 19078 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:12:09.579594 19078 webserver.cc:460] Webserver started at http://127.31.156.65:42499/ using document root <none> and password file <none>
I20260812 07:12:09.580513 19078 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:09.580657 19078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:09.587121 19078 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:09.590997 19142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:09.593068 19078 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:09.593279 19078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "8fb049cebf9644efabdec069ffd1ba8b"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "6f14d200dcc99d717f6083f976cf49e1"
server_key_iv: "d1e8bed248a2c2026d1fba4a7ccea77b"
server_key_version: "encryptionkey@0"
I20260812 07:12:09.594362 19078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:09.622834 19078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:09.627739 19078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:09.628000 19078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:09.628894 19078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:09.630560 19149 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 07:12:09.635425 19078 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 07:12:09.635545 19078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 07:12:09.635630 19078 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 07:12:09.638370 19078 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 07:12:09.638461 19078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260812 07:12:09.638909 19149 tablet_bootstrap.cc:492] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b: Bootstrap starting.
I20260812 07:12:09.651907 19078 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:35271
I20260812 07:12:09.652281 19274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:35271 every 8 connection(s)
I20260812 07:12:09.653644 19078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:09.659709 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 19078
I20260812 07:12:09.660338 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:36725
--local_ip_for_outbound_sockets=127.31.156.66
--tserver_master_addrs=127.31.156.126:45227
--webserver_port=38261
--webserver_interface=127.31.156.66
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:12:09.662302 19149 log.cc:824] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:09.670462 19149 tablet_bootstrap.cc:492] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:12:09.671037 19149 tablet_bootstrap.cc:492] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b: Bootstrap complete.
I20260812 07:12:09.672629 19149 ts_tablet_manager.cc:1397] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b: Time spent bootstrapping tablet: real 0.034s user 0.028s sys 0.003s
I20260812 07:12:09.677462 19149 raft_consensus.cc:348] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:12:09.678282 19149 raft_consensus.cc:729] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb049cebf9644efabdec069ffd1ba8b, State: Initialized, Role: FOLLOWER
I20260812 07:12:09.679023 19149 consensus_queue.cc:260] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:12:09.680470 19149 ts_tablet_manager.cc:1428] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
W20260812 07:12:09.719317 19277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:12:09.656450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:55671)
0812 07:12:09.657341 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:09.657422 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:09.658840 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:09.718320 (+ 59480us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":353,"threads_started":1}
W20260812 07:12:09.720079 19275 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:45227 (0 consecutive failures): Network error: Failed to ping master at 127.31.156.126:45227: Client connection negotiation failed: client connection to 127.31.156.126:45227: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 07:12:09.721006 19277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:09.720683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:57489)
0812 07:12:09.720833 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:09.720932 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 07:12:09.721870 19277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:09.721612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:51505)
0812 07:12:09.721740 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:09.721806 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 07:12:09.722102 19275 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:12:09.801270 19278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:09.801620 19278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:09.801714 19278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:09.806186 19278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:09.806300 19278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:09.806408 19278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:12:09.812134 19278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:36725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=38261
--tserver_master_addrs=127.31.156.126:45227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:09.813411 19278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:09.814515 19278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:09.825747 19286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:09.825747 19289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:09.825749 19287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:09.826172 19278 server_base.cc:1034] running on GCE node
I20260812 07:12:09.826771 19278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:09.827469 19278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:09.828739 19278 hybrid_clock.cc:648] HybridClock initialized: now 1786518729828700 us; error 59 us; skew 500 ppm
Aug 12 07:12:09 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518729, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:09.832406 19278 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:12:09.833622 19278 webserver.cc:460] Webserver started at http://127.31.156.66:38261/ using document root <none> and password file <none>
I20260812 07:12:09.834273 19278 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:09.834380 19278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:09.839578 19278 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:12:09.842106 19300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:09.844079 19278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:09.844293 19278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "ddd8d839b30941f39f757b56296514f8"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "a776cab3e01594e2ca18a31c5d71dbe1"
server_key_iv: "0e29f6cd1cc666aa525182bd1e5d1e06"
server_key_version: "encryptionkey@0"
I20260812 07:12:09.845009 19278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:09.859990 19278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:09.864071 19278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:09.864336 19278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:09.865084 19278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:09.866099 19278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:09.866158 19278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:09.866217 19278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:09.866242 19278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:09.877759 19278 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:36725
I20260812 07:12:09.877784 19430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:36725 every 8 connection(s)
I20260812 07:12:09.879024 19278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:12:09.883368 19434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:09.881478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:52759)
0812 07:12:09.882011 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:09.882493 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":132,"threads_started":1}
W20260812 07:12:09.884124 19431 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:45227 (0 consecutive failures): Network error: Failed to ping master at 127.31.156.126:45227: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
I20260812 07:12:09.885131 19434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:09.884842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:35095)
0812 07:12:09.884984 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:09.885049 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 07:12:09.886099 19434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:09.885878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:58179)
0812 07:12:09.886000 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:09.886046 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 07:12:09.886408 19431 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:12:09.887688 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 19278
I20260812 07:12:09.888072 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:42863
--local_ip_for_outbound_sockets=127.31.156.67
--tserver_master_addrs=127.31.156.126:45227
--webserver_port=43561
--webserver_interface=127.31.156.67
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:10.016538 19435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:10.016847 19435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:10.016894 19435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:10.021941 19435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:10.022028 19435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:10.022096 19435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:12:10.028697 19435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:42863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=43561
--tserver_master_addrs=127.31.156.126:45227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:10.029996 19435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:10.031188 19435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:10.043910 19443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:10.044000 19441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:10.044574 19447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:10.045014 19435 server_base.cc:1034] running on GCE node
I20260812 07:12:10.045571 19435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:10.046861 19435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:10.050227 19435 hybrid_clock.cc:648] HybridClock initialized: now 1786518730050191 us; error 51 us; skew 500 ppm
Aug 12 07:12:10 dist-test-slave-csg5 krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518730, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:10.054832 19435 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:12:10.056468 19435 webserver.cc:460] Webserver started at http://127.31.156.67:43561/ using document root <none> and password file <none>
I20260812 07:12:10.057366 19435 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:10.057471 19435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:10.064067 19435 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:10.067140 19455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:10.069201 19435 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:12:10.069392 19435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "1057199d80f447ce9013ca00e6758b81"
format_stamp: "Formatted at 2026-08-12 07:11:27 on dist-test-slave-csg5"
server_key: "dde7b80e2d3741d7fceca89a7fa41676"
server_key_iv: "bdcd00831e56605d752eb8f9d9e66d79"
server_key_version: "encryptionkey@0"
I20260812 07:12:10.070331 19435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:10.092727 19435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:10.097023 19435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:10.097276 19435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:10.097990 19435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:10.099009 19435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:10.099061 19435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:10.099140 19435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:10.099261 19435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:10.109522 19435 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:42863
I20260812 07:12:10.109541 19581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:42863 every 8 connection(s)
I20260812 07:12:10.110641 19435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:10.115065 19584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:10.113148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:50275)
0812 07:12:10.113640 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:10.113755 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":120,"threads_started":1}
W20260812 07:12:10.115671 19582 heartbeater.cc:644] Failed to heartbeat to 127.31.156.126:45227 (0 consecutive failures): Network error: Failed to ping master at 127.31.156.126:45227: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
I20260812 07:12:10.116349 19584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:10.116110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:59797)
0812 07:12:10.116232 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:10.116290 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 07:12:10.117161 19584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:10.116934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:34047)
0812 07:12:10.117054 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:10.117108 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 07:12:10.117385 19582 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:12:10.117671 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 19435
I20260812 07:12:10.723475 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:10.722915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:54653)
0812 07:12:10.723274 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:10.723350 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":87,"threads_started":1}
I20260812 07:12:10.824499 19589 raft_consensus.cc:482] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:10.824797 19589 raft_consensus.cc:504] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:12:10.826170 19589 leader_election.cc:288] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863), ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725)
Aug 12 07:12:10 dist-test-slave-csg5 krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518729, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.67@KRBTEST.COM
Aug 12 07:12:10 dist-test-slave-csg5 krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518729, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.66@KRBTEST.COM
I20260812 07:12:10.845527 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:10.826847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:42863 (local address 127.31.156.65:45733)
0812 07:12:10.827165 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:10.827195 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:10.827418 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:10.829491 (+ 2073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:10.829498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:10.830079 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:10.830563 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:10.830582 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:10.832327 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:10.832329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:10.833334 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:10.833347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:10.834277 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:10.834281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:10.834510 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:10.835676 (+ 1166us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:10.835688 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:10.837598 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:10.841250 (+ 3652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:10.841259 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:10.841280 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:10.841952 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:10.842616 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:10.842620 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:10.842622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:10.842956 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:10.844037 (+ 1081us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:10.844049 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:10.844502 (+ 453us) client_negotiation.cc:769] Sending connection context
0812 07:12:10.844758 (+ 256us) client_negotiation.cc:240] Negotiation successful
0812 07:12:10.845339 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":296,"spinlock_wait_cycles":64768}
I20260812 07:12:10.845606 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:10.827086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:45733 (local address 127.31.156.67:42863)
0812 07:12:10.827757 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:10.827776 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:10.827828 (+ 52us) server_negotiation.cc:407] Connection header received
0812 07:12:10.828654 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:10.828661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:10.829049 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:10.829395 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:10.830896 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:10.832140 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:10.833489 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:10.834099 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:10.834318 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:10.838034 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:10.838046 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:10.838068 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:10.838117 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:10.841002 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:10.842159 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:10.842165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:10.842170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:10.842400 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:10.843123 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:10.843127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:10.843129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:10.843859 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:10.844003 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:10.845025 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0812 07:12:10.845398 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":153,"threads_started":1}
I20260812 07:12:10.845655 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:10.827724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:36743 (local address 127.31.156.66:36725)
0812 07:12:10.828127 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:10.828145 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:10.828176 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:12:10.828799 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:10.828805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:10.829122 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:10.829454 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:10.830960 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:10.832022 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:10.833592 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:10.834089 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:10.834290 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:10.838430 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:10.838438 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:10.838453 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:10.838497 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:10.841086 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:10.842151 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:10.842160 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:10.842167 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:10.842466 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:10.843109 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:10.843113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:10.843115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:10.843827 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:10.844003 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:10.845025 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0812 07:12:10.845482 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":136,"threads_started":1}
I20260812 07:12:10.845527 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:10.827401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:36725 (local address 127.31.156.65:36743)
0812 07:12:10.827867 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:10.827879 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:10.828027 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:10.829492 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:10.829498 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:10.830327 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:10.830776 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:10.830791 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:10.832218 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:10.832224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:10.833399 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:10.833444 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:10.834242 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:10.834245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:10.834542 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:10.835603 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:10.835623 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:10.838012 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:10.841250 (+ 3238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:10.841259 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:10.841280 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:10.841952 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:10.842616 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:10.842620 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:10.842622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:10.842956 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:10.844044 (+ 1088us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:10.844049 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:10.844502 (+ 453us) client_negotiation.cc:769] Sending connection context
0812 07:12:10.844758 (+ 256us) client_negotiation.cc:240] Negotiation successful
0812 07:12:10.845257 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":249,"spinlock_wait_cycles":34816,"thread_start_us":90,"threads_started":1}
W20260812 07:12:10.848886 19143 leader_election.cc:341] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863): Not found: Tablet not found: 8c7ccc976b51436e93637480ec71f88f
W20260812 07:12:10.849210 19143 leader_election.cc:341] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725): Not found: Tablet not found: 8c7ccc976b51436e93637480ec71f88f
I20260812 07:12:10.849310 19143 leader_election.cc:302] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fb049cebf9644efabdec069ffd1ba8b; no voters: 1057199d80f447ce9013ca00e6758b81, ddd8d839b30941f39f757b56296514f8
I20260812 07:12:10.849644 19589 raft_consensus.cc:2726] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:12:10.887647 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:10.887133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:35907)
0812 07:12:10.887464 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:10.887528 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":85,"threads_started":1}
I20260812 07:12:11.118580 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:11.118062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:52753)
0812 07:12:11.118400 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:11.118466 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":88,"threads_started":1}
I20260812 07:12:11.725167 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:11.724538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:45403)
0812 07:12:11.724978 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:11.725046 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":151,"threads_started":1}
I20260812 07:12:11.889012 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:11.888525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:56493)
0812 07:12:11.888821 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:11.888886 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":83,"threads_started":1}
I20260812 07:12:12.120410 19599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:12.119832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:40217)
0812 07:12:12.120122 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:12.120244 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":81,"threads_started":1}
I20260812 07:12:12.716620 19600 raft_consensus.cc:482] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:12.716785 19600 raft_consensus.cc:504] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:12:12.717383 19600 leader_election.cc:288] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863), ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725)
W20260812 07:12:12.718504 19143 leader_election.cc:341] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725): Not found: Tablet not found: 8c7ccc976b51436e93637480ec71f88f
W20260812 07:12:12.718703 19143 leader_election.cc:341] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863): Not found: Tablet not found: 8c7ccc976b51436e93637480ec71f88f
I20260812 07:12:12.718802 19143 leader_election.cc:302] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fb049cebf9644efabdec069ffd1ba8b; no voters: 1057199d80f447ce9013ca00e6758b81, ddd8d839b30941f39f757b56296514f8
I20260812 07:12:12.719048 19600 raft_consensus.cc:2726] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:12:12.726930 19602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:12.726365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:47381)
0812 07:12:12.726729 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:12.726816 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":154,"threads_started":1}
I20260812 07:12:12.890479 19603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:12.889937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:46767)
0812 07:12:12.890291 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:12.890365 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":112,"threads_started":1}
I20260812 07:12:13.121997 19604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:13.121474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:36561)
0812 07:12:13.121818 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:13.121887 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":113,"threads_started":1}
I20260812 07:12:13.728438 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:13.727849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:43761)
0812 07:12:13.728193 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:13.728327 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":118,"threads_started":1}
I20260812 07:12:13.891928 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:13.891453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:51705)
0812 07:12:13.891739 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:13.891818 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":183,"threads_started":1}
I20260812 07:12:14.123641 19607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:14.123106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:44359)
0812 07:12:14.123434 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:14.123508 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":81,"threads_started":1}
I20260812 07:12:14.730270 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:14.729706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:35517)
0812 07:12:14.730084 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:14.730152 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":111,"threads_started":1}
I20260812 07:12:14.893512 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:14.892983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:51679)
0812 07:12:14.893336 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:14.893407 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":85,"threads_started":1}
I20260812 07:12:15.125306 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:15.124663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:34191)
0812 07:12:15.125011 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:15.125083 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":101,"threads_started":1}
I20260812 07:12:15.305061 19611 raft_consensus.cc:482] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:15.305313 19611 raft_consensus.cc:504] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:12:15.305915 19611 leader_election.cc:288] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863), ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725)
W20260812 07:12:15.307265 19143 leader_election.cc:341] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863): Not found: Tablet not found: 8c7ccc976b51436e93637480ec71f88f
W20260812 07:12:15.307466 19143 leader_election.cc:341] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725): Not found: Tablet not found: 8c7ccc976b51436e93637480ec71f88f
I20260812 07:12:15.307530 19143 leader_election.cc:302] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fb049cebf9644efabdec069ffd1ba8b; no voters: 1057199d80f447ce9013ca00e6758b81, ddd8d839b30941f39f757b56296514f8
I20260812 07:12:15.307708 19611 raft_consensus.cc:2726] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:12:15.731745 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:15.731224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:41479)
0812 07:12:15.731578 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:15.731636 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":85,"threads_started":1}
I20260812 07:12:15.895139 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:15.894568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:60277)
0812 07:12:15.894959 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:15.895030 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":171,"threads_started":1}
I20260812 07:12:16.127084 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:16.126462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:49025)
0812 07:12:16.126898 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:16.126969 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":94,"threads_started":1}
I20260812 07:12:16.733403 19616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:16.732805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:57911)
0812 07:12:16.733110 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:16.733279 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":80,"threads_started":1}
I20260812 07:12:16.896966 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:16.896286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:60779)
0812 07:12:16.896713 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:16.896821 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":94,"threads_started":1}
I20260812 07:12:17.014911 19618 raft_consensus.cc:482] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:17.015076 19618 raft_consensus.cc:504] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:12:17.015965 19618 leader_election.cc:288] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863), ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725)
W20260812 07:12:17.017035 19143 leader_election.cc:341] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863): Not found: Tablet not found: 8c7ccc976b51436e93637480ec71f88f
W20260812 07:12:17.017187 19143 leader_election.cc:341] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725): Not found: Tablet not found: 8c7ccc976b51436e93637480ec71f88f
I20260812 07:12:17.017387 19143 leader_election.cc:302] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fb049cebf9644efabdec069ffd1ba8b; no voters: 1057199d80f447ce9013ca00e6758b81, ddd8d839b30941f39f757b56296514f8
I20260812 07:12:17.017585 19618 raft_consensus.cc:2726] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:12:17.128958 19619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:17.128334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:47619)
0812 07:12:17.128765 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:17.128842 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":107,"threads_started":1}
I20260812 07:12:17.734941 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:17.734361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:49777)
0812 07:12:17.734751 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:17.734823 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":83,"threads_started":1}
I20260812 07:12:17.898748 19622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:17.898079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:48455)
0812 07:12:17.898455 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:17.898579 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":112,"threads_started":1}
I20260812 07:12:18.130872 19624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:18.130134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:47199)
0812 07:12:18.130546 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:18.130621 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":122,"threads_started":1}
I20260812 07:12:18.736423 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:18.735920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:51709)
0812 07:12:18.736228 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:18.736295 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":86,"threads_started":1}
I20260812 07:12:18.900338 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:18.899794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:41499)
0812 07:12:18.900140 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:18.900212 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":108,"threads_started":1}
I20260812 07:12:19.132481 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:19.131953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:60657)
0812 07:12:19.132286 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:19.132360 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":110,"threads_started":1}
I20260812 07:12:19.738353 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:19.737786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:47731)
0812 07:12:19.738126 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:19.738227 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":82,"threads_started":1}
I20260812 07:12:19.902107 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:19.901534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:55083)
0812 07:12:19.901918 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:19.901990 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":159,"threads_started":1}
I20260812 07:12:20.134074 19630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:20.133475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:52787)
0812 07:12:20.133824 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:20.133885 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":118,"threads_started":1}
I20260812 07:12:20.740022 19632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:20.739539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:51607)
0812 07:12:20.739843 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:20.739914 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":88,"threads_started":1}
I20260812 07:12:20.903600 19633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:20.903079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:34921)
0812 07:12:20.903417 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:20.903487 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":88,"threads_started":1}
I20260812 07:12:21.135833 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:21.135318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:55065)
0812 07:12:21.135648 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:21.135723 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":123,"threads_started":1}
I20260812 07:12:21.741645 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:21.741106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:46877)
0812 07:12:21.741457 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:21.741529 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":100,"threads_started":1}
I20260812 07:12:21.905073 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:21.904578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:43551)
0812 07:12:21.904883 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:21.904964 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":87,"threads_started":1}
I20260812 07:12:22.137312 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:22.136807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:39623)
0812 07:12:22.137093 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:22.137178 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":106,"threads_started":1}
I20260812 07:12:22.743381 19638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:22.742833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:36669)
0812 07:12:22.743154 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:22.743225 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":84,"threads_started":1}
I20260812 07:12:22.906638 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:22.906142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:58135)
0812 07:12:22.906456 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:22.906529 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":83,"threads_started":1}
I20260812 07:12:23.138800 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:23.138223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:53471)
0812 07:12:23.138541 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:23.138611 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":117,"threads_started":1}
I20260812 07:12:23.745011 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:23.744474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:40997)
0812 07:12:23.744830 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:23.744902 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":107,"threads_started":1}
I20260812 07:12:23.891088 19644 raft_consensus.cc:482] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:23.891297 19644 raft_consensus.cc:504] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057199d80f447ce9013ca00e6758b81" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 42863 } } peers { permanent_uuid: "ddd8d839b30941f39f757b56296514f8" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 36725 } } peers { permanent_uuid: "8fb049cebf9644efabdec069ffd1ba8b" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35271 } }
I20260812 07:12:23.891806 19644 leader_election.cc:288] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863), ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725)
W20260812 07:12:23.892860 19143 leader_election.cc:341] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddd8d839b30941f39f757b56296514f8 (127.31.156.66:36725): Not found: Tablet not found: 8c7ccc976b51436e93637480ec71f88f
W20260812 07:12:23.893050 19143 leader_election.cc:341] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1057199d80f447ce9013ca00e6758b81 (127.31.156.67:42863): Not found: Tablet not found: 8c7ccc976b51436e93637480ec71f88f
I20260812 07:12:23.893155 19143 leader_election.cc:302] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fb049cebf9644efabdec069ffd1ba8b; no voters: 1057199d80f447ce9013ca00e6758b81, ddd8d839b30941f39f757b56296514f8
I20260812 07:12:23.893404 19644 raft_consensus.cc:2726] T 8c7ccc976b51436e93637480ec71f88f P 8fb049cebf9644efabdec069ffd1ba8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:12:23.908095 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:23.907580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:54787)
0812 07:12:23.907906 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:23.907976 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":87,"threads_started":1}
I20260812 07:12:24.140583 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:24.139935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:60723)
0812 07:12:24.140396 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:24.140467 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":162,"threads_started":1}
I20260812 07:12:24.746924 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:24.746250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.65:45151)
0812 07:12:24.746623 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:24.746785 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":98,"threads_started":1}
I20260812 07:12:24.909667 19650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:24.909121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.66:50083)
0812 07:12:24.909505 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:24.909563 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":184,"threads_started":1}
I20260812 07:12:25.121488 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 19078
I20260812 07:12:25.132735 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 19278
I20260812 07:12:25.142108 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:25.141665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:45227 (local address 127.31.156.67:39143)
0812 07:12:25.141947 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:25.142006 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.126:45227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":103,"threads_started":1}
I20260812 07:12:25.142414 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 19435
2026-08-12T07:12:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59818 ms)
[----------] 30 tests from SecurityITest (276573 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:25 dist-test-slave-csg5 krb5kdc[19654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:25 dist-test-slave-csg5 krb5kdc[19654](info): set up 2 sockets
Aug 12 07:12:25 dist-test-slave-csg5 krb5kdc[19654](info): commencing operation
krb5kdc: starting...
W20260812 07:12:26.356446 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:26 dist-test-slave-csg5 krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:12:26.516109 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:40471
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:40471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:26.650519 19668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:26.650956 19668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:26.651047 19668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:26.655346 19668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:26.655475 19668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:26.655514 19668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:26.655557 19668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:26.655596 19668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:26.662758 19668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:40471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:40471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:26.664153 19668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:26.665364 19668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:26.676158 19674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:26.676164 19678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:26.676577 19668 server_base.cc:1034] running on GCE node
W20260812 07:12:26.676158 19673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:26.677420 19668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:26.678853 19668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:26.680070 19668 hybrid_clock.cc:648] HybridClock initialized: now 1786518746680042 us; error 51 us; skew 500 ppm
Aug 12 07:12:26 dist-test-slave-csg5 krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518746, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:26.683606 19668 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:12:26.685024 19668 webserver.cc:460] Webserver started at http://127.31.156.126:34323/ using document root <none> and password file <none>
I20260812 07:12:26.685891 19668 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:26.686019 19668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:26.686326 19668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:26.688964 19668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "836325ad4cad4a1eb9a35ddd70ef7055"
format_stamp: "Formatted at 2026-08-12 07:12:26 on dist-test-slave-csg5"
server_key: "20e30a2f419f2c7a504d4c83da274792"
server_key_iv: "e075d7ed103eaa2456b67eed6286ef2f"
server_key_version: "encryptionkey@0"
I20260812 07:12:26.689721 19668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "836325ad4cad4a1eb9a35ddd70ef7055"
format_stamp: "Formatted at 2026-08-12 07:12:26 on dist-test-slave-csg5"
server_key: "20e30a2f419f2c7a504d4c83da274792"
server_key_iv: "e075d7ed103eaa2456b67eed6286ef2f"
server_key_version: "encryptionkey@0"
I20260812 07:12:26.694645 19668 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:12:26.697917 19687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:26.699627 19668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:26.699834 19668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "836325ad4cad4a1eb9a35ddd70ef7055"
format_stamp: "Formatted at 2026-08-12 07:12:26 on dist-test-slave-csg5"
server_key: "20e30a2f419f2c7a504d4c83da274792"
server_key_iv: "e075d7ed103eaa2456b67eed6286ef2f"
server_key_version: "encryptionkey@0"
I20260812 07:12:26.699993 19668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:26.719988 19668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:26.723951 19668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:26.724207 19668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:26.732627 19668 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:40471
I20260812 07:12:26.732710 19748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:40471 every 8 connection(s)
I20260812 07:12:26.733759 19668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:26.736050 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 19668
I20260812 07:12:26.736214 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:26.736553 32369 external_mini_cluster.cc:1428] Setting key 0ac920056bb506507a6766a9f00d6db8
I20260812 07:12:26.737459 19749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:26.744817 19749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055: Bootstrap starting.
Aug 12 07:12:26 dist-test-slave-csg5 krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:12:26.747900 19749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:26.748703 19749 log.cc:824] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:26.751487 19749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055: No bootstrap required, opened a new log
I20260812 07:12:26.754534 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:26.738235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47866 (local address 127.31.156.126:40471)
0812 07:12:26.738801 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:26.738820 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:26.738861 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:12:26.739599 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:26.739626 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:26.739991 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:26.740326 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:26.741329 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:26.742532 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:26.743804 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:26.744381 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:26.744520 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:26.747923 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:26.747930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:26.747947 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:26.747984 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:26.750416 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:26.751105 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:26.751112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:26.751119 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:26.751283 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:26.751789 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:26.751793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:26.751794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:26.752414 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:26.752622 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:26.752973 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 07:12:26.753321 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":189,"threads_started":1}
I20260812 07:12:26.756435 19749 raft_consensus.cc:348] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "836325ad4cad4a1eb9a35ddd70ef7055" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40471 } }
I20260812 07:12:26.756755 19749 raft_consensus.cc:374] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:26.756808 19749 raft_consensus.cc:729] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 836325ad4cad4a1eb9a35ddd70ef7055, State: Initialized, Role: FOLLOWER
I20260812 07:12:26.757263 19749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "836325ad4cad4a1eb9a35ddd70ef7055" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40471 } }
I20260812 07:12:26.757431 19749 raft_consensus.cc:388] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:26.757500 19749 raft_consensus.cc:482] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:26.757630 19749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:26.758749 19749 raft_consensus.cc:504] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "836325ad4cad4a1eb9a35ddd70ef7055" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40471 } }
I20260812 07:12:26.759145 19749 leader_election.cc:302] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 836325ad4cad4a1eb9a35ddd70ef7055; no voters:
I20260812 07:12:26.759521 19749 leader_election.cc:288] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:26.759737 19755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:26.760016 19755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [term 1 LEADER]: Becoming Leader. State: Replica: 836325ad4cad4a1eb9a35ddd70ef7055, State: Running, Role: LEADER
I20260812 07:12:26.760397 19755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "836325ad4cad4a1eb9a35ddd70ef7055" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40471 } }
I20260812 07:12:26.761031 19749 sys_catalog.cc:564] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:26.762384 19757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 836325ad4cad4a1eb9a35ddd70ef7055. Latest consensus state: current_term: 1 leader_uuid: "836325ad4cad4a1eb9a35ddd70ef7055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "836325ad4cad4a1eb9a35ddd70ef7055" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40471 } } }
I20260812 07:12:26.762524 19757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:26.762828 19756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "836325ad4cad4a1eb9a35ddd70ef7055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "836325ad4cad4a1eb9a35ddd70ef7055" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 40471 } } }
I20260812 07:12:26.762943 19756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055 [sys.catalog]: This master's current role is: LEADER
W20260812 07:12:26.766969 19771 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:12:26.767091 19771 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:12:26.767189 19765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:26.768041 19765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:26.774868 19765 catalog_manager.cc:1347] Generated new cluster ID: c6af78d10d594f0993a02b351a4a3986
I20260812 07:12:26.774982 19765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:26.794817 19765 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:26.796036 19765 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:26.801908 19765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055: Generated new TSK 0
I20260812 07:12:26.802799 19765 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:12:26.851256 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40471
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:26.983767 19777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:26.984081 19777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:26.984129 19777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:26.988436 19777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:26.988520 19777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:26.988585 19777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:12:26.994267 19777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:26.995549 19777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:26.996644 19777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:27.007682 19785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:27.007655 19783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:27.007663 19782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:27.007786 19777 server_base.cc:1034] running on GCE node
I20260812 07:12:27.008522 19777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:27.009325 19777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:27.010564 19777 hybrid_clock.cc:648] HybridClock initialized: now 1786518747010520 us; error 52 us; skew 500 ppm
Aug 12 07:12:27 dist-test-slave-csg5 krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518747, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:27.014772 19777 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:12:27.015974 19777 webserver.cc:460] Webserver started at http://127.31.156.65:34241/ using document root <none> and password file <none>
I20260812 07:12:27.016737 19777 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:27.016857 19777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:27.017120 19777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:27.019526 19777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "4553aed0414c42779e9f4bb696c6091d"
format_stamp: "Formatted at 2026-08-12 07:12:27 on dist-test-slave-csg5"
server_key: "735514ba34a5d8917c64cce1c560b0c5"
server_key_iv: "957dba83cc381e6071cb1afb899bba5a"
server_key_version: "encryptionkey@0"
I20260812 07:12:27.020242 19777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "4553aed0414c42779e9f4bb696c6091d"
format_stamp: "Formatted at 2026-08-12 07:12:27 on dist-test-slave-csg5"
server_key: "735514ba34a5d8917c64cce1c560b0c5"
server_key_iv: "957dba83cc381e6071cb1afb899bba5a"
server_key_version: "encryptionkey@0"
I20260812 07:12:27.024940 19777 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:12:27.028010 19794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:27.029551 19777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:12:27.029752 19777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "4553aed0414c42779e9f4bb696c6091d"
format_stamp: "Formatted at 2026-08-12 07:12:27 on dist-test-slave-csg5"
server_key: "735514ba34a5d8917c64cce1c560b0c5"
server_key_iv: "957dba83cc381e6071cb1afb899bba5a"
server_key_version: "encryptionkey@0"
I20260812 07:12:27.029911 19777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:27.052661 19777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:27.056877 19777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:27.057121 19777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:27.057792 19777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:27.058730 19777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:27.058779 19777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:27.058825 19777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:27.058883 19777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:27.069064 19777 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:43185
I20260812 07:12:27.069088 19926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:43185 every 8 connection(s)
I20260812 07:12:27.070098 19777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:27.071455 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 19777
I20260812 07:12:27.071600 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:12:27.071995 32369 external_mini_cluster.cc:1428] Setting key 597f3e901e8ff2bb564ee6cbef4a9aef
Aug 12 07:12:27 dist-test-slave-csg5 krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518747, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
I20260812 07:12:27.090975 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.072951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:54003 (local address 127.31.156.126:40471)
0812 07:12:27.073142 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:27.073147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:27.074319 (+ 1172us) server_negotiation.cc:407] Connection header received
0812 07:12:27.076804 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:27.076810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:27.076887 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:27.077001 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:27.078508 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.079546 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.081528 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.082046 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.082210 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.085670 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:27.085674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:27.085677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:27.085714 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:27.087449 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:27.088175 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:27.088178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:27.088179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:27.088263 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:27.088983 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:27.088988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:27.088990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:27.089381 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:27.089479 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:27.090356 (+ 877us) server_negotiation.cc:299] Negotiation successful
0812 07:12:27.090493 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:12:27.092218 19929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.073428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40471 (local address 127.31.156.65:54003)
0812 07:12:27.074100 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:27.074177 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:27.076444 (+ 2267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:27.077206 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:27.077214 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:27.077662 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:27.078304 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.078321 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.079758 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.079761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.081321 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.081338 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.082298 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.082303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.082492 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.083371 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:27.083392 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:27.085419 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:27.087598 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:27.087605 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:27.087632 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:27.088071 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:27.088531 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:27.088534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:27.088537 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:27.088850 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:27.089539 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:27.089550 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:27.089996 (+ 446us) client_negotiation.cc:769] Sending connection context
0812 07:12:27.090299 (+ 303us) client_negotiation.cc:240] Negotiation successful
0812 07:12:27.090879 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":186,"threads_started":1}
I20260812 07:12:27.093991 19927 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40471
I20260812 07:12:27.094303 19927 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:27.094944 19927 heartbeater.cc:511] Master 127.31.156.126:40471 requested a full tablet report, sending...
I20260812 07:12:27.096992 19704 ts_manager.cc:194] Registered new tserver with Master: 4553aed0414c42779e9f4bb696c6091d (127.31.156.65:43185)
I20260812 07:12:27.098814 19704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:54003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:12:27.112013 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40471
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:27.237960 19934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:27.238319 19934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:27.238377 19934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:27.242626 19934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:27.242740 19934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:27.242805 19934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:12:27.248607 19934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:27.249872 19934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:27.251022 19934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:27.262145 19940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:27.262799 19939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:27.262816 19942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:27.263136 19934 server_base.cc:1034] running on GCE node
I20260812 07:12:27.264031 19934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:27.264740 19934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:27.265941 19934 hybrid_clock.cc:648] HybridClock initialized: now 1786518747265909 us; error 47 us; skew 500 ppm
Aug 12 07:12:27 dist-test-slave-csg5 krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518747, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:27.269584 19934 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:12:27.271009 19934 webserver.cc:460] Webserver started at http://127.31.156.66:44965/ using document root <none> and password file <none>
I20260812 07:12:27.271695 19934 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:27.271819 19934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:27.272094 19934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:27.274814 19934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "9ffbcffa8ab94a47b0779320283ca816"
format_stamp: "Formatted at 2026-08-12 07:12:27 on dist-test-slave-csg5"
server_key: "c0df64b49f09f5a04ed8081d9438972d"
server_key_iv: "0510a3fc4aaadaaebdd0073992191f6d"
server_key_version: "encryptionkey@0"
I20260812 07:12:27.275552 19934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "9ffbcffa8ab94a47b0779320283ca816"
format_stamp: "Formatted at 2026-08-12 07:12:27 on dist-test-slave-csg5"
server_key: "c0df64b49f09f5a04ed8081d9438972d"
server_key_iv: "0510a3fc4aaadaaebdd0073992191f6d"
server_key_version: "encryptionkey@0"
I20260812 07:12:27.280400 19934 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260812 07:12:27.283921 19950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:27.285465 19934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:12:27.285665 19934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "9ffbcffa8ab94a47b0779320283ca816"
format_stamp: "Formatted at 2026-08-12 07:12:27 on dist-test-slave-csg5"
server_key: "c0df64b49f09f5a04ed8081d9438972d"
server_key_iv: "0510a3fc4aaadaaebdd0073992191f6d"
server_key_version: "encryptionkey@0"
I20260812 07:12:27.285837 19934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:27.305756 19934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:27.310010 19934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:27.310300 19934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:27.311118 19934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:27.312446 19934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:27.312531 19934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:27.312616 19934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:27.312665 19934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:27.324580 19934 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:34333
I20260812 07:12:27.324613 20081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:34333 every 8 connection(s)
I20260812 07:12:27.325698 19934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:12:27.332290 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 19934
I20260812 07:12:27.332460 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:12:27.332820 32369 external_mini_cluster.cc:1428] Setting key eaf54e9eb523df8a64f22237be12bd07
Aug 12 07:12:27 dist-test-slave-csg5 krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518747, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:12:27.345142 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.328048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:51723 (local address 127.31.156.126:40471)
0812 07:12:27.328264 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:27.328269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:27.329246 (+ 977us) server_negotiation.cc:407] Connection header received
0812 07:12:27.331035 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:27.331039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:27.331107 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:27.331211 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:27.333363 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.334171 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.336532 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.336806 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.336917 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.340914 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:27.340918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:27.340920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:27.340960 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:27.342618 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:27.343338 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:27.343342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:27.343344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:27.343434 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:27.343837 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:27.343841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:27.343843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:27.344060 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:27.344206 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:27.344797 (+ 591us) server_negotiation.cc:299] Negotiation successful
0812 07:12:27.344950 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 07:12:27.346082 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.328462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40471 (local address 127.31.156.66:51723)
0812 07:12:27.329046 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:27.329101 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:27.330753 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:27.331474 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:27.331483 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:27.332113 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:27.333072 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.333103 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.334371 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.334374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.336311 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.336333 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.337014 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.337022 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.337299 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.338149 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:27.338169 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:27.340623 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:27.342840 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:27.342848 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:27.342870 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:27.343193 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:27.343538 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:27.343541 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:27.343544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:27.343720 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:27.344177 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:27.344184 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:27.344529 (+ 345us) client_negotiation.cc:769] Sending connection context
0812 07:12:27.344737 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 07:12:27.345138 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":155,"threads_started":1}
I20260812 07:12:27.347527 20082 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40471
I20260812 07:12:27.347815 20082 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:27.348533 20082 heartbeater.cc:511] Master 127.31.156.126:40471 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
I20260812 07:12:27.349872 19704 ts_manager.cc:194] Registered new tserver with Master: 9ffbcffa8ab94a47b0779320283ca816 (127.31.156.66:34333)
I20260812 07:12:27.350548 19704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:51723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:12:27.371126 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40471
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:27.507517 20088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:27.507880 20088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:27.507931 20088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:27.512632 20088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:27.512718 20088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:27.512784 20088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:12:27.518980 20088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:40471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:27.520244 20088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:27.521364 20088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:27.531998 20094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:27.531994 20096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:27.532406 20099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:27.532452 20088 server_base.cc:1034] running on GCE node
I20260812 07:12:27.532935 20088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:27.533893 20088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:27.535118 20088 hybrid_clock.cc:648] HybridClock initialized: now 1786518747534994 us; error 138 us; skew 500 ppm
Aug 12 07:12:27 dist-test-slave-csg5 krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518747, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:27.538851 20088 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:12:27.540098 20088 webserver.cc:460] Webserver started at http://127.31.156.67:43417/ using document root <none> and password file <none>
I20260812 07:12:27.540823 20088 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:27.540935 20088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:27.541236 20088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:27.544319 20088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "26bc9011008c46ada78e73ade208036b"
format_stamp: "Formatted at 2026-08-12 07:12:27 on dist-test-slave-csg5"
server_key: "ca14d561d66699a632d12dc24feb8ed3"
server_key_iv: "2defb7d2038a077f13c90d0d01afca4d"
server_key_version: "encryptionkey@0"
I20260812 07:12:27.545145 20088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "26bc9011008c46ada78e73ade208036b"
format_stamp: "Formatted at 2026-08-12 07:12:27 on dist-test-slave-csg5"
server_key: "ca14d561d66699a632d12dc24feb8ed3"
server_key_iv: "2defb7d2038a077f13c90d0d01afca4d"
server_key_version: "encryptionkey@0"
I20260812 07:12:27.550602 20088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:12:27.554647 20108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:27.556833 20088 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:12:27.557052 20088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "26bc9011008c46ada78e73ade208036b"
format_stamp: "Formatted at 2026-08-12 07:12:27 on dist-test-slave-csg5"
server_key: "ca14d561d66699a632d12dc24feb8ed3"
server_key_iv: "2defb7d2038a077f13c90d0d01afca4d"
server_key_version: "encryptionkey@0"
I20260812 07:12:27.557237 20088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:27.579406 20088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:27.584025 20088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:27.584316 20088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:27.585081 20088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:27.586207 20088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:27.586316 20088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:27.586407 20088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:27.586464 20088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:27.598434 20088 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:45613
I20260812 07:12:27.598502 20241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:45613 every 8 connection(s)
I20260812 07:12:27.599572 20088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:27.601181 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 20088
I20260812 07:12:27.601446 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:12:27.601809 32369 external_mini_cluster.cc:1428] Setting key e03eff4bfc4cb38c18fb07e865c1a4f9
Aug 12 07:12:27 dist-test-slave-csg5 krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518747, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:12:27.616823 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.601867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:51307 (local address 127.31.156.126:40471)
0812 07:12:27.602042 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:27.602046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:27.603145 (+ 1099us) server_negotiation.cc:407] Connection header received
0812 07:12:27.604890 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:27.604893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:27.604984 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:27.605059 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:27.606501 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.607151 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.608421 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.608695 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.608820 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.612044 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:27.612047 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:27.612049 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:27.612080 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:27.613706 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:27.614599 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:27.614604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:27.614606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:27.614749 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:27.615299 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:27.615302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:27.615305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:27.615598 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:27.615692 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:27.616492 (+ 800us) server_negotiation.cc:299] Negotiation successful
0812 07:12:27.616649 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:12:27.617825 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.602343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:40471 (local address 127.31.156.67:51307)
0812 07:12:27.602965 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:27.603032 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:27.604667 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:27.605315 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:27.605325 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:27.605768 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:27.606331 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.606347 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.607297 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.607301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.608274 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.608285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.608893 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.608897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.609091 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.609830 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:27.609849 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:27.611801 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:27.613866 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:27.614026 (+ 160us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:27.614047 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:27.614464 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:27.614953 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:27.614957 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:27.614959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:27.615199 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:27.615706 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:27.615714 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:27.616069 (+ 355us) client_negotiation.cc:769] Sending connection context
0812 07:12:27.616412 (+ 343us) client_negotiation.cc:240] Negotiation successful
0812 07:12:27.616861 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":146,"threads_started":1}
I20260812 07:12:27.619174 20242 heartbeater.cc:347] Connected to a master server at 127.31.156.126:40471
I20260812 07:12:27.619426 20242 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:27.619968 20242 heartbeater.cc:511] Master 127.31.156.126:40471 requested a full tablet report, sending...
I20260812 07:12:27.621284 19704 ts_manager.cc:194] Registered new tserver with Master: 26bc9011008c46ada78e73ade208036b (127.31.156.67:45613)
I20260812 07:12:27.621965 19704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:51307
I20260812 07:12:27.629796 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:12:27.642397 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.632565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.31.156.126:40471)
0812 07:12:27.632719 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:27.632723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:27.632740 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:12:27.632910 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:27.632914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:27.632966 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:27.633077 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:27.634171 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.635163 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.636275 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.636672 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.636812 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.638704 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:27.638709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:27.638712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:27.638749 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:27.640458 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:27.640964 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:27.640968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:27.640970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:27.641063 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:27.641368 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:27.641372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:27.641374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:27.641645 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:27.641796 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:27.642073 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 07:12:27.642212 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:12:27.646420 19704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:12:27.649951 19704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:12:27.668041 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.661650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36208 (local address 127.31.156.66:34333)
0812 07:12:27.662216 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:27.662227 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:27.662448 (+ 221us) server_negotiation.cc:407] Connection header received
0812 07:12:27.662544 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:27.662550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:27.662812 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:27.663033 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:27.663832 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.664944 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.666544 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.667247 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.667385 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.667560 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:27.667763 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 07:12:27.667868 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":172,"threads_started":1}
I20260812 07:12:27.668033 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.660780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:34333 (local address 127.0.0.1:36208)
0812 07:12:27.662253 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:27.662270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:27.662371 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:27.662998 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:27.663005 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:27.663032 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:27.663573 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.663583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.665150 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.665153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.666386 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.666398 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.667392 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.667395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.667523 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.667564 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 07:12:27.667718 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 07:12:27.667868 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1395,"mutex_wait_us":291,"thread_start_us":97,"threads_started":1}
I20260812 07:12:27.669741 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.660980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47146 (local address 127.31.156.67:45613)
0812 07:12:27.662701 (+ 1721us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:27.662714 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:27.662737 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:12:27.662812 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:27.662817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:27.662993 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:27.663251 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:27.663514 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.665289 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.667532 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.669056 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.669176 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.669322 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:27.669467 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 07:12:27.669576 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1573,"thread_start_us":182,"threads_started":1}
I20260812 07:12:27.669682 20254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.660780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:45613 (local address 127.0.0.1:47146)
0812 07:12:27.661513 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:27.661586 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:27.661760 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:27.663147 (+ 1387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:27.663150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:27.663169 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:27.663343 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.663353 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.665444 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.665449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.667346 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.667360 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.669202 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.669206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.669308 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.669347 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 07:12:27.669430 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 07:12:27.669513 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":330,"thread_start_us":118,"threads_started":1}
I20260812 07:12:27.669873 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.661646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:43185 (local address 127.0.0.1:48840)
0812 07:12:27.662510 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:27.662528 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:27.662634 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:27.663449 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:27.663453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:27.663474 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:27.663645 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.663655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.665215 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.665219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.667346 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.667360 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.669385 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.669392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.669491 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.669511 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 07:12:27.669566 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 07:12:27.669629 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":284,"thread_start_us":124,"threads_started":1}
I20260812 07:12:27.670465 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.662124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48840 (local address 127.31.156.65:43185)
0812 07:12:27.662560 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:27.662570 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:27.662599 (+ 29us) server_negotiation.cc:407] Connection header received
0812 07:12:27.662807 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:27.662812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:27.663076 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:27.663288 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:27.663755 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.665037 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.667532 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.668798 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.669131 (+ 333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.670123 (+ 992us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:27.670171 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 07:12:27.670265 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":110,"threads_started":1}
I20260812 07:12:27.672330 20010 tablet_service.cc:1467] Processing CreateTablet for tablet 2edd98b446e9426f997d290a70ea21ac (DEFAULT_TABLE table=test-table [id=e4b874fb903f4ae09bf589a562c9f72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:27.673197 19851 tablet_service.cc:1467] Processing CreateTablet for tablet 2edd98b446e9426f997d290a70ea21ac (DEFAULT_TABLE table=test-table [id=e4b874fb903f4ae09bf589a562c9f72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:27.673195 20167 tablet_service.cc:1467] Processing CreateTablet for tablet 2edd98b446e9426f997d290a70ea21ac (DEFAULT_TABLE table=test-table [id=e4b874fb903f4ae09bf589a562c9f72f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:27.673767 20010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2edd98b446e9426f997d290a70ea21ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:27.674160 19851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2edd98b446e9426f997d290a70ea21ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:27.674497 20167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2edd98b446e9426f997d290a70ea21ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:27.680913 20262 tablet_bootstrap.cc:492] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d: Bootstrap starting.
I20260812 07:12:27.683105 20264 tablet_bootstrap.cc:492] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816: Bootstrap starting.
I20260812 07:12:27.683702 20265 tablet_bootstrap.cc:492] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b: Bootstrap starting.
I20260812 07:12:27.684641 20262 tablet_bootstrap.cc:654] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:27.685672 20262 log.cc:824] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:27.685860 20264 tablet_bootstrap.cc:654] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:27.686079 20265 tablet_bootstrap.cc:654] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:27.687413 20264 log.cc:824] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:27.687517 20265 log.cc:824] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:27.688596 20262 tablet_bootstrap.cc:492] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d: No bootstrap required, opened a new log
I20260812 07:12:27.688865 20262 ts_tablet_manager.cc:1397] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:12:27.690016 20264 tablet_bootstrap.cc:492] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816: No bootstrap required, opened a new log
I20260812 07:12:27.690267 20264 ts_tablet_manager.cc:1397] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260812 07:12:27.692277 20265 tablet_bootstrap.cc:492] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b: No bootstrap required, opened a new log
I20260812 07:12:27.692550 20265 ts_tablet_manager.cc:1397] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260812 07:12:27.694988 20262 raft_consensus.cc:348] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffbcffa8ab94a47b0779320283ca816" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34333 } } peers { permanent_uuid: "26bc9011008c46ada78e73ade208036b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45613 } } peers { permanent_uuid: "4553aed0414c42779e9f4bb696c6091d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 43185 } }
I20260812 07:12:27.695016 20264 raft_consensus.cc:348] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffbcffa8ab94a47b0779320283ca816" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34333 } } peers { permanent_uuid: "26bc9011008c46ada78e73ade208036b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45613 } } peers { permanent_uuid: "4553aed0414c42779e9f4bb696c6091d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 43185 } }
I20260812 07:12:27.695348 20262 raft_consensus.cc:374] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:27.695348 20264 raft_consensus.cc:374] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:27.695426 20264 raft_consensus.cc:729] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffbcffa8ab94a47b0779320283ca816, State: Initialized, Role: FOLLOWER
I20260812 07:12:27.695456 20262 raft_consensus.cc:729] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4553aed0414c42779e9f4bb696c6091d, State: Initialized, Role: FOLLOWER
I20260812 07:12:27.696002 20264 consensus_queue.cc:260] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffbcffa8ab94a47b0779320283ca816" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34333 } } peers { permanent_uuid: "26bc9011008c46ada78e73ade208036b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45613 } } peers { permanent_uuid: "4553aed0414c42779e9f4bb696c6091d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 43185 } }
I20260812 07:12:27.696002 20262 consensus_queue.cc:260] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffbcffa8ab94a47b0779320283ca816" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34333 } } peers { permanent_uuid: "26bc9011008c46ada78e73ade208036b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45613 } } peers { permanent_uuid: "4553aed0414c42779e9f4bb696c6091d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 43185 } }
I20260812 07:12:27.695967 20265 raft_consensus.cc:348] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffbcffa8ab94a47b0779320283ca816" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34333 } } peers { permanent_uuid: "26bc9011008c46ada78e73ade208036b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45613 } } peers { permanent_uuid: "4553aed0414c42779e9f4bb696c6091d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 43185 } }
I20260812 07:12:27.696209 20265 raft_consensus.cc:374] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:27.696286 20265 raft_consensus.cc:729] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26bc9011008c46ada78e73ade208036b, State: Initialized, Role: FOLLOWER
I20260812 07:12:27.696782 20265 consensus_queue.cc:260] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffbcffa8ab94a47b0779320283ca816" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34333 } } peers { permanent_uuid: "26bc9011008c46ada78e73ade208036b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45613 } } peers { permanent_uuid: "4553aed0414c42779e9f4bb696c6091d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 43185 } }
I20260812 07:12:27.697165 19927 heartbeater.cc:503] Master 127.31.156.126:40471 was elected leader, sending a full tablet report...
I20260812 07:12:27.697237 20262 ts_tablet_manager.cc:1428] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260812 07:12:27.697395 20264 ts_tablet_manager.cc:1428] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816: Time spent starting tablet: real 0.007s user 0.009s sys 0.000s
I20260812 07:12:27.697892 20242 heartbeater.cc:503] Master 127.31.156.126:40471 was elected leader, sending a full tablet report...
I20260812 07:12:27.697979 20265 ts_tablet_manager.cc:1428] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 07:12:27.698567 20082 heartbeater.cc:503] Master 127.31.156.126:40471 was elected leader, sending a full tablet report...
I20260812 07:12:27.706094 20274 raft_consensus.cc:482] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:27.706393 20274 raft_consensus.cc:504] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffbcffa8ab94a47b0779320283ca816" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34333 } } peers { permanent_uuid: "26bc9011008c46ada78e73ade208036b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45613 } } peers { permanent_uuid: "4553aed0414c42779e9f4bb696c6091d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 43185 } }
I20260812 07:12:27.707652 20274 leader_election.cc:288] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ffbcffa8ab94a47b0779320283ca816 (127.31.156.66:34333), 4553aed0414c42779e9f4bb696c6091d (127.31.156.65:43185)
I20260812 07:12:27.713061 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.707982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:34333 (local address 127.31.156.67:53367)
0812 07:12:27.708144 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:27.708161 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:27.708269 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:27.708986 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:27.708989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:27.709006 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:27.709139 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.709145 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.709922 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.709927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.711136 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.711147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.712465 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.712468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.712642 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.712713 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 07:12:27.712793 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 07:12:27.712876 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":56}
I20260812 07:12:27.713133 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.708108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:53367 (local address 127.31.156.66:34333)
0812 07:12:27.708599 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:27.708604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:27.708630 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:12:27.708693 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:27.708696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:27.708747 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:27.708829 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:27.709268 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.709793 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.711539 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.712303 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.712459 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.712519 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:27.712853 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 07:12:27.712949 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394}
I20260812 07:12:27.714332 20034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edd98b446e9426f997d290a70ea21ac" candidate_uuid: "26bc9011008c46ada78e73ade208036b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ffbcffa8ab94a47b0779320283ca816" is_pre_election: true
I20260812 07:12:27.714457 20275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.708443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:43185 (local address 127.31.156.67:59115)
0812 07:12:27.708850 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:27.708867 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:27.708975 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:27.709953 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:27.709956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:27.709982 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:27.710137 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.710168 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.711231 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.711489 (+ 258us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.712900 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:27.712913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.713961 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.713964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:27.714057 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.714079 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 07:12:27.714187 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 07:12:27.714273 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":37,"thread_start_us":165,"threads_started":1}
I20260812 07:12:27.714582 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.708564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:59115 (local address 127.31.156.65:43185)
0812 07:12:27.708795 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:27.708802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:27.709001 (+ 199us) server_negotiation.cc:407] Connection header received
0812 07:12:27.709069 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:27.709072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:27.709122 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:27.709220 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:27.710333 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.711087 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.713083 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.713825 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.713939 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.713979 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:27.714346 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 07:12:27.714443 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:12:27.714759 20034 raft_consensus.cc:2454] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26bc9011008c46ada78e73ade208036b in term 0.
I20260812 07:12:27.715363 20113 leader_election.cc:302] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26bc9011008c46ada78e73ade208036b, 9ffbcffa8ab94a47b0779320283ca816; no voters:
I20260812 07:12:27.715394 19876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edd98b446e9426f997d290a70ea21ac" candidate_uuid: "26bc9011008c46ada78e73ade208036b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4553aed0414c42779e9f4bb696c6091d" is_pre_election: true
I20260812 07:12:27.715739 19876 raft_consensus.cc:2454] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26bc9011008c46ada78e73ade208036b in term 0.
I20260812 07:12:27.715728 20274 raft_consensus.cc:2781] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:12:27.715847 20274 raft_consensus.cc:482] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:12:27.715906 20274 raft_consensus.cc:3037] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:27.717074 20274 raft_consensus.cc:504] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffbcffa8ab94a47b0779320283ca816" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34333 } } peers { permanent_uuid: "26bc9011008c46ada78e73ade208036b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45613 } } peers { permanent_uuid: "4553aed0414c42779e9f4bb696c6091d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 43185 } }
I20260812 07:12:27.717429 20274 leader_election.cc:288] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [CANDIDATE]: Term 1 election: Requested vote from peers 9ffbcffa8ab94a47b0779320283ca816 (127.31.156.66:34333), 4553aed0414c42779e9f4bb696c6091d (127.31.156.65:43185)
I20260812 07:12:27.717986 20034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edd98b446e9426f997d290a70ea21ac" candidate_uuid: "26bc9011008c46ada78e73ade208036b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ffbcffa8ab94a47b0779320283ca816"
I20260812 07:12:27.718125 20034 raft_consensus.cc:3037] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:27.718290 19876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edd98b446e9426f997d290a70ea21ac" candidate_uuid: "26bc9011008c46ada78e73ade208036b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4553aed0414c42779e9f4bb696c6091d"
I20260812 07:12:27.718462 19876 raft_consensus.cc:3037] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:27.719492 20034 raft_consensus.cc:2454] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26bc9011008c46ada78e73ade208036b in term 1.
I20260812 07:12:27.719691 19876 raft_consensus.cc:2454] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26bc9011008c46ada78e73ade208036b in term 1.
I20260812 07:12:27.719946 20113 leader_election.cc:302] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26bc9011008c46ada78e73ade208036b, 9ffbcffa8ab94a47b0779320283ca816; no voters:
I20260812 07:12:27.720148 20274 raft_consensus.cc:2781] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:27.720407 20274 raft_consensus.cc:686] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 1 LEADER]: Becoming Leader. State: Replica: 26bc9011008c46ada78e73ade208036b, State: Running, Role: LEADER
I20260812 07:12:27.720785 20274 consensus_queue.cc:237] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffbcffa8ab94a47b0779320283ca816" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34333 } } peers { permanent_uuid: "26bc9011008c46ada78e73ade208036b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45613 } } peers { permanent_uuid: "4553aed0414c42779e9f4bb696c6091d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 43185 } }
I20260812 07:12:27.724789 19704 catalog_manager.cc:5495] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b reported cstate change: term changed from 0 to 1, leader changed from <none> to 26bc9011008c46ada78e73ade208036b (127.31.156.67). New cstate: current_term: 1 leader_uuid: "26bc9011008c46ada78e73ade208036b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffbcffa8ab94a47b0779320283ca816" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26bc9011008c46ada78e73ade208036b" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4553aed0414c42779e9f4bb696c6091d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 43185 } health_report { overall_health: UNKNOWN } } }
I20260812 07:12:27.750782 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.746547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47156 (local address 127.31.156.67:45613)
0812 07:12:27.746756 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:27.746761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:27.746775 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:12:27.746856 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:27.746859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:27.746920 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:27.747065 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:12:27.747374 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.747960 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.748990 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:27.749251 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:27.749374 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:27.749762 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:12:27.750239 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:12:27.750390 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:27.750463 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 07:12:27.750531 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:12:27.757808 19876 raft_consensus.cc:1261] T 2edd98b446e9426f997d290a70ea21ac P 4553aed0414c42779e9f4bb696c6091d [term 1 FOLLOWER]: Refusing update from remote peer 26bc9011008c46ada78e73ade208036b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:27.757797 20034 raft_consensus.cc:1261] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816 [term 1 FOLLOWER]: Refusing update from remote peer 26bc9011008c46ada78e73ade208036b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:27.758638 20276 consensus_queue.cc:1035] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4553aed0414c42779e9f4bb696c6091d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:27.758924 20274 consensus_queue.cc:1035] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ffbcffa8ab94a47b0779320283ca816" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:27.777667 19705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:27.777915 19705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:27.781136 19705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 836325ad4cad4a1eb9a35ddd70ef7055: Sending DeleteTablet for 3 replicas of tablet 2edd98b446e9426f997d290a70ea21ac
I20260812 07:12:27.782133 20167 tablet_service.cc:1514] Processing DeleteTablet for tablet 2edd98b446e9426f997d290a70ea21ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:27 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:47146
I20260812 07:12:27.782326 20010 tablet_service.cc:1514] Processing DeleteTablet for tablet 2edd98b446e9426f997d290a70ea21ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:27 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:36208
I20260812 07:12:27.782346 19851 tablet_service.cc:1514] Processing DeleteTablet for tablet 2edd98b446e9426f997d290a70ea21ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:27 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:48840
I20260812 07:12:27.782755 20291 tablet_replica.cc:330] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b: stopping tablet replica
I20260812 07:12:27.783147 20291 raft_consensus.cc:2229] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:12:27.783320 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 19777
I20260812 07:12:27.783604 20291 raft_consensus.cc:2258] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:27.790144 20292 tablet_replica.cc:330] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816: stopping tablet replica
I20260812 07:12:27.791049 20292 raft_consensus.cc:2229] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:27.791783 20292 raft_consensus.cc:2258] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:27.792487 20291 ts_tablet_manager.cc:1905] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:27.796891 20292 ts_tablet_manager.cc:1905] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:27.798326 20291 ts_tablet_manager.cc:1918] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:27.798601 20291 log.cc:1196] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/2edd98b446e9426f997d290a70ea21ac
I20260812 07:12:27.799155 20291 ts_tablet_manager.cc:1939] T 2edd98b446e9426f997d290a70ea21ac P 26bc9011008c46ada78e73ade208036b: Deleting consensus metadata
I20260812 07:12:27.800956 19688 catalog_manager.cc:4841] TS 26bc9011008c46ada78e73ade208036b (127.31.156.67:45613): tablet 2edd98b446e9426f997d290a70ea21ac (table test-table [id=e4b874fb903f4ae09bf589a562c9f72f]) successfully deleted
I20260812 07:12:27.801434 20292 ts_tablet_manager.cc:1918] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:27.801827 20292 log.cc:1196] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/2edd98b446e9426f997d290a70ea21ac
I20260812 07:12:27.802376 20292 ts_tablet_manager.cc:1939] T 2edd98b446e9426f997d290a70ea21ac P 9ffbcffa8ab94a47b0779320283ca816: Deleting consensus metadata
W20260812 07:12:27.802717 19690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:43185 (error 108)
I20260812 07:12:27.803869 19690 catalog_manager.cc:4841] TS 9ffbcffa8ab94a47b0779320283ca816 (127.31.156.66:34333): tablet 2edd98b446e9426f997d290a70ea21ac (table test-table [id=e4b874fb903f4ae09bf589a562c9f72f]) successfully deleted
I20260812 07:12:27.803995 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:27.803647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:43185 (local address 127.0.0.1:48848)
0812 07:12:27.803795 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:27.803877 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:43185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 07:12:27.804121 19690 catalog_manager.cc:4584] TS 4553aed0414c42779e9f4bb696c6091d (127.31.156.65:43185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2edd98b446e9426f997d290a70ea21ac: Network error: Client connection negotiation failed: client connection to 127.31.156.65:43185: connect: Connection refused (error 111)
I20260812 07:12:27.804656 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 19934
I20260812 07:12:27.820271 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 20088
I20260812 07:12:27.832829 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 19668
2026-08-12T07:12:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2690 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T07:12:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:27Z Disabled control of system clock
I20260812 07:12:27.872704 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:35195
--webserver_interface=127.31.156.126
--webserver_port=0
--builtin_ntp_servers=127.31.156.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:35195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:12:28.001830 20299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:28.002203 20299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:28.002245 20299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:28.006757 20299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:28.006850 20299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:28.006865 20299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:28.006883 20299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:28.006899 20299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:28.012082 20299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:35195
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:35195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:28.013290 20299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:28.014926 20299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:12:28.024680 20299 server_base.cc:1034] running on GCE node
W20260812 07:12:28.024740 20307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:28.024658 20304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:28.024662 20305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:28.025810 20299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:28.027199 20299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:28.028437 20299 hybrid_clock.cc:648] HybridClock initialized: now 1786518748028416 us; error 39 us; skew 500 ppm
I20260812 07:12:28.031131 20299 webserver.cc:460] Webserver started at http://127.31.156.126:33973/ using document root <none> and password file <none>
I20260812 07:12:28.031885 20299 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:28.031965 20299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:28.032253 20299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:28.034789 20299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "3d12459da2834192967f01717c81320a"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "d7fb457a91c3355c92efb7f1a5f68898"
server_key_iv: "22ad80d6afde3e3d5fa06a4d1344db04"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.035503 20299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "3d12459da2834192967f01717c81320a"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "d7fb457a91c3355c92efb7f1a5f68898"
server_key_iv: "22ad80d6afde3e3d5fa06a4d1344db04"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.040107 20299 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:28.043314 20313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:28.044963 20299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:12:28.045097 20299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "3d12459da2834192967f01717c81320a"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "d7fb457a91c3355c92efb7f1a5f68898"
server_key_iv: "22ad80d6afde3e3d5fa06a4d1344db04"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.045290 20299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:28.073537 20299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:28.074373 20299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:28.074594 20299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:28.082839 20299 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:35195
I20260812 07:12:28.082960 20372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:35195 every 8 connection(s)
I20260812 07:12:28.083999 20299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:28.087402 20374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:28.091329 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 20299
I20260812 07:12:28.091449 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:28.091791 32369 external_mini_cluster.cc:1428] Setting key fdd16f50bbe91f76b8c59ddb8fdca2b2
I20260812 07:12:28.094280 20374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a: Bootstrap starting.
I20260812 07:12:28.096819 20374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:28.097682 20374 log.cc:824] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:28.099926 20374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a: No bootstrap required, opened a new log
I20260812 07:12:28.102133 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.093493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.31.156.126:35195)
0812 07:12:28.094248 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.094274 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.094314 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:12:28.095128 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.095150 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.095660 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.096036 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:28.096531 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.097638 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.098643 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.099201 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.099410 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.100324 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:28.100330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:28.100345 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:28.100765 (+ 420us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:28.100826 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:28.100847 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:28.100964 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.101086 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.101379 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":152,"threads_started":1}
I20260812 07:12:28.104270 20374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d12459da2834192967f01717c81320a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35195 } }
I20260812 07:12:28.104578 20374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:28.104677 20374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d12459da2834192967f01717c81320a, State: Initialized, Role: FOLLOWER
I20260812 07:12:28.105198 20374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d12459da2834192967f01717c81320a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35195 } }
I20260812 07:12:28.105350 20374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:28.105422 20374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:28.105552 20374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:28.106639 20374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d12459da2834192967f01717c81320a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35195 } }
I20260812 07:12:28.107080 20374 leader_election.cc:302] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d12459da2834192967f01717c81320a; no voters:
I20260812 07:12:28.107460 20374 leader_election.cc:288] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:28.107542 20381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:28.107751 20381 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [term 1 LEADER]: Becoming Leader. State: Replica: 3d12459da2834192967f01717c81320a, State: Running, Role: LEADER
I20260812 07:12:28.108096 20381 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d12459da2834192967f01717c81320a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35195 } }
I20260812 07:12:28.108914 20374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:28.110086 20382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d12459da2834192967f01717c81320a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d12459da2834192967f01717c81320a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35195 } } }
I20260812 07:12:28.110122 20384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d12459da2834192967f01717c81320a. Latest consensus state: current_term: 1 leader_uuid: "3d12459da2834192967f01717c81320a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d12459da2834192967f01717c81320a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35195 } } }
I20260812 07:12:28.110252 20382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:28.110251 20384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:28.110854 20393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:28.113818 20393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:28.120585 20393 catalog_manager.cc:1347] Generated new cluster ID: dedef1604f264f03932df083fea83c0d
I20260812 07:12:28.120712 20393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:28.131953 20393 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:28.133728 20393 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:28.141148 20393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a: Generated new TSK 0
I20260812 07:12:28.142135 20393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:28.152709 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:35195
--builtin_ntp_servers=127.31.156.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:12:28.283550 20405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:28.283895 20405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:28.283941 20405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:28.288398 20405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:28.288483 20405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:28.288545 20405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:12:28.293742 20405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:28.295145 20405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:28.296736 20405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:28.308106 20412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:28.308187 20411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:28.308195 20414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:28.309517 20405 server_base.cc:1034] running on GCE node
I20260812 07:12:28.310328 20405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:28.311316 20405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:28.312563 20405 hybrid_clock.cc:648] HybridClock initialized: now 1786518748312520 us; error 62 us; skew 500 ppm
I20260812 07:12:28.314874 20405 webserver.cc:460] Webserver started at http://127.31.156.65:36963/ using document root <none> and password file <none>
I20260812 07:12:28.315572 20405 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:28.315680 20405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:28.315948 20405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:28.318311 20405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "d5522f334b014ece971e3a4a5a7826cf"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "9beb4e4c9be202f875b5050ef59930eb"
server_key_iv: "39de65a063e10ad263e3ffd68814b093"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.319007 20405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "d5522f334b014ece971e3a4a5a7826cf"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "9beb4e4c9be202f875b5050ef59930eb"
server_key_iv: "39de65a063e10ad263e3ffd68814b093"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.323310 20405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:12:28.326098 20421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:28.327656 20405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:28.327857 20405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "d5522f334b014ece971e3a4a5a7826cf"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "9beb4e4c9be202f875b5050ef59930eb"
server_key_iv: "39de65a063e10ad263e3ffd68814b093"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.327991 20405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:28.341848 20405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:28.342638 20405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:28.342938 20405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:28.343591 20405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:28.344646 20405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:28.344729 20405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:28.344827 20405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:28.344878 20405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:28.357043 20405 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:34989
I20260812 07:12:28.357131 20559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:34989 every 8 connection(s)
I20260812 07:12:28.358187 20405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:28.361953 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 20405
I20260812 07:12:28.362082 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:12:28.362370 32369 external_mini_cluster.cc:1428] Setting key b1c16466b1c828d25f9f2f24dfb31ac1
I20260812 07:12:28.364861 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:35195
--builtin_ntp_servers=127.31.156.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:12:28.370056 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.360498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:50573 (local address 127.31.156.126:35195)
0812 07:12:28.360644 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.360648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.361821 (+ 1173us) server_negotiation.cc:407] Connection header received
0812 07:12:28.363549 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.363552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.363597 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.363669 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:28.364922 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.365551 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.367328 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.367738 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.367875 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.369288 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:28.369295 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:28.369301 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:28.369323 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:28.369352 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:28.369366 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:28.369454 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.369725 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.369816 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:12:28.371217 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.360851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:35195 (local address 127.31.156.65:50573)
0812 07:12:28.361618 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:28.361684 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:28.363306 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:28.363883 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:28.363891 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:28.363982 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:28.364746 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.364770 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.365694 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.365701 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.367171 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.367183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.367942 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.367946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.368156 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.368958 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:12:28.368960 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:28.369009 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:12:28.369091 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:28.369493 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:28.369504 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:28.369509 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:12:28.369709 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 07:12:28.370008 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":136,"threads_started":1}
I20260812 07:12:28.373004 20560 heartbeater.cc:347] Connected to a master server at 127.31.156.126:35195
I20260812 07:12:28.373339 20560 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:28.373958 20560 heartbeater.cc:511] Master 127.31.156.126:35195 requested a full tablet report, sending...
I20260812 07:12:28.375650 20330 ts_manager.cc:194] Registered new tserver with Master: d5522f334b014ece971e3a4a5a7826cf (127.31.156.65:34989)
I20260812 07:12:28.376932 20330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.65:50573
W20260812 07:12:28.491639 20563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:28.491992 20563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:28.492038 20563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:28.496466 20563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:28.496562 20563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:28.496624 20563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:12:28.502308 20563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:28.503867 20563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:28.505513 20563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:12:28.517256 20563 server_base.cc:1034] running on GCE node
W20260812 07:12:28.517191 20570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:28.517175 20573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:28.517175 20569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:28.517970 20563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:28.518807 20563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:28.519999 20563 hybrid_clock.cc:648] HybridClock initialized: now 1786518748519968 us; error 42 us; skew 500 ppm
I20260812 07:12:28.522195 20563 webserver.cc:460] Webserver started at http://127.31.156.66:35031/ using document root <none> and password file <none>
I20260812 07:12:28.522954 20563 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:28.523027 20563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:28.523267 20563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:28.525571 20563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "e12a5db66593442090e0675b1b4a640e"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "c1a8a89fb624d46893db55cdb8215e91"
server_key_iv: "a5bb970869a5021c776acfe7a4909a17"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.526265 20563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "e12a5db66593442090e0675b1b4a640e"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "c1a8a89fb624d46893db55cdb8215e91"
server_key_iv: "a5bb970869a5021c776acfe7a4909a17"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.530604 20563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:12:28.534070 20579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:28.535660 20563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:28.535781 20563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "e12a5db66593442090e0675b1b4a640e"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "c1a8a89fb624d46893db55cdb8215e91"
server_key_iv: "a5bb970869a5021c776acfe7a4909a17"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.535967 20563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:28.555666 20563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:28.556488 20563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:28.556708 20563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:28.557399 20563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:28.558441 20563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:28.558493 20563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:28.558537 20563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:28.558553 20563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:28.569535 20563 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:46301
I20260812 07:12:28.569556 20710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:46301 every 8 connection(s)
I20260812 07:12:28.570585 20563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:12:28.574729 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 20563
I20260812 07:12:28.574867 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:12:28.575318 32369 external_mini_cluster.cc:1428] Setting key eb8282b59c0efe42b9f17fe7920b74bb
I20260812 07:12:28.578248 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:35195
--builtin_ntp_servers=127.31.156.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:12:28.583772 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.572943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:60921 (local address 127.31.156.126:35195)
0812 07:12:28.573157 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.573163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.574519 (+ 1356us) server_negotiation.cc:407] Connection header received
0812 07:12:28.576464 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.576469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.576539 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.576652 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:28.578045 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.579042 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.580994 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.581270 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.581386 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.583104 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:28.583108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:28.583110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:28.583128 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:28.583153 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:28.583162 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:28.583264 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.583531 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.583604 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 07:12:28.585301 20713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.573399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:35195 (local address 127.31.156.66:60921)
0812 07:12:28.574316 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:28.574367 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:28.576159 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:28.576929 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:28.576940 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:28.577072 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:28.577852 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.577868 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.579251 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.579256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.580799 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.580816 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.581483 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.581491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.581790 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.582673 (+ 883us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:12:28.582678 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:28.582751 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:12:28.582867 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:28.583283 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:28.583298 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:28.583310 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 07:12:28.583476 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 07:12:28.583900 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":226,"threads_started":1}
I20260812 07:12:28.586930 20711 heartbeater.cc:347] Connected to a master server at 127.31.156.126:35195
I20260812 07:12:28.587230 20711 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:28.587878 20711 heartbeater.cc:511] Master 127.31.156.126:35195 requested a full tablet report, sending...
I20260812 07:12:28.589186 20330 ts_manager.cc:194] Registered new tserver with Master: e12a5db66593442090e0675b1b4a640e (127.31.156.66:46301)
I20260812 07:12:28.589869 20330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.66:60921
W20260812 07:12:28.707897 20714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:28.708304 20714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:28.708391 20714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:28.713637 20714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:28.713742 20714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:28.713850 20714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:12:28.719945 20714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:28.721345 20714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:28.723093 20714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:28.734987 20722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:28.735044 20721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:28.735225 20714 server_base.cc:1034] running on GCE node
W20260812 07:12:28.735028 20724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:28.735998 20714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:28.736784 20714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:28.737962 20714 hybrid_clock.cc:648] HybridClock initialized: now 1786518748737941 us; error 38 us; skew 500 ppm
I20260812 07:12:28.740360 20714 webserver.cc:460] Webserver started at http://127.31.156.67:35489/ using document root <none> and password file <none>
I20260812 07:12:28.741055 20714 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:28.741132 20714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:28.741369 20714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:28.743692 20714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "9ad50cf546eb424990e3e9de8e021210"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "b7d09c7b36b914384e92d8712d518726"
server_key_iv: "80162d7d3e78f6c891861860685de2b5"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.744333 20714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "9ad50cf546eb424990e3e9de8e021210"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "b7d09c7b36b914384e92d8712d518726"
server_key_iv: "80162d7d3e78f6c891861860685de2b5"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.748891 20714 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:12:28.751724 20730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:28.753158 20714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:28.753341 20714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "9ad50cf546eb424990e3e9de8e021210"
format_stamp: "Formatted at 2026-08-12 07:12:28 on dist-test-slave-csg5"
server_key: "b7d09c7b36b914384e92d8712d518726"
server_key_iv: "80162d7d3e78f6c891861860685de2b5"
server_key_version: "encryptionkey@0"
I20260812 07:12:28.753473 20714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:28.764698 20714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:28.765463 20714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:28.765688 20714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:28.766322 20714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:28.767418 20714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:28.767493 20714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:28.767571 20714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:28.767625 20714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:28.776928 20714 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:34993
I20260812 07:12:28.776999 20870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:34993 every 8 connection(s)
I20260812 07:12:28.778017 20714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:28.778306 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 20714
I20260812 07:12:28.778417 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:12:28.778798 32369 external_mini_cluster.cc:1428] Setting key 9dfab6511c933e1264b8f25b077bad0c
I20260812 07:12:28.789945 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.780611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:47861 (local address 127.31.156.126:35195)
0812 07:12:28.780804 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.780810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.782211 (+ 1401us) server_negotiation.cc:407] Connection header received
0812 07:12:28.783797 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.783800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.783858 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.783940 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:28.785054 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.785812 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.787477 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.787730 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.787862 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.789367 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:28.789371 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:28.789373 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:28.789395 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:28.789419 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:28.789428 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:28.789494 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.789722 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.789812 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 07:12:28.791109 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.781163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:35195 (local address 127.31.156.67:47861)
0812 07:12:28.782048 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:28.782101 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:28.783540 (+ 1439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:28.784123 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:28.784131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:28.784264 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:28.784868 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.784883 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.785994 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.785998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.787327 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.787341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.787952 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.787955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.788170 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.789030 (+ 860us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:12:28.789031 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:28.789083 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:12:28.789169 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:28.789545 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:28.789556 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:28.789562 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:12:28.789717 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 07:12:28.789992 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":135,"threads_started":1}
I20260812 07:12:28.792471 20871 heartbeater.cc:347] Connected to a master server at 127.31.156.126:35195
I20260812 07:12:28.792762 20871 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:28.793515 20871 heartbeater.cc:511] Master 127.31.156.126:35195 requested a full tablet report, sending...
I20260812 07:12:28.794698 20330 ts_manager.cc:194] Registered new tserver with Master: 9ad50cf546eb424990e3e9de8e021210 (127.31.156.67:34993)
I20260812 07:12:28.795349 20330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.67:47861
I20260812 07:12:28.806355 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:12:28.813954 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.808893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34528 (local address 127.31.156.126:35195)
0812 07:12:28.809069 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.809074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.809456 (+ 382us) server_negotiation.cc:407] Connection header received
0812 07:12:28.809511 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.809513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.809562 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.809646 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:28.810072 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.810912 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.811942 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.812222 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.812375 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.813409 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:28.813414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:28.813416 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:28.813433 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:28.813463 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:28.813473 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:28.813564 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.813706 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.813781 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:12:28.817948 20330 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:12:28.821538 20330 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:12:28.838207 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.831432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:46301 (local address 127.0.0.1:50950)
0812 07:12:28.831873 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:28.831932 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:28.832082 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:28.833339 (+ 1257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:28.833347 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:28.833373 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:28.833542 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.833549 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.835104 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.835107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.836605 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.836617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.837664 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.837668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.837762 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.837786 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:12:28.837854 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 07:12:28.838002 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":187,"thread_start_us":111,"threads_started":1}
I20260812 07:12:28.838495 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.831635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50950 (local address 127.31.156.66:46301)
0812 07:12:28.832218 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.832231 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.832285 (+ 54us) server_negotiation.cc:407] Connection header received
0812 07:12:28.832370 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.832376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.833139 (+ 763us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.833322 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:28.833671 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.834956 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.836752 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.837514 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.837955 (+ 441us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.838129 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.838184 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.838293 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":140,"threads_started":1}
I20260812 07:12:28.838691 20892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.832610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34989 (local address 127.0.0.1:39464)
0812 07:12:28.833347 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:28.833361 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:28.833449 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:28.833981 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:28.833984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:28.834010 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:28.834097 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.834103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.835472 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.835475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.836685 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.836697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.838261 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.838265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.838356 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.838382 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 07:12:28.838464 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 07:12:28.838531 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":27,"thread_start_us":92,"threads_started":1}
I20260812 07:12:28.838207 20890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.832264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:34993 (local address 127.0.0.1:57828)
0812 07:12:28.832630 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:28.832645 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:28.832744 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:28.833490 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:28.833493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:28.833508 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:28.833590 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.833598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.834979 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.834983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.836234 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.836245 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.837508 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.837512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.837644 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.837687 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 07:12:28.837810 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 07:12:28.838001 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":146,"thread_start_us":112,"threads_started":1}
I20260812 07:12:28.839159 20891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.832533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57828 (local address 127.31.156.67:34993)
0812 07:12:28.832995 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.833004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.833024 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:12:28.833132 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.833136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.833331 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.833736 (+ 405us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:28.833848 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.834860 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.836510 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.837355 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.837506 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.837725 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.837869 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.839000 (+ 1131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":118,"threads_started":1}
I20260812 07:12:28.839303 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.832796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.31.156.65:34989)
0812 07:12:28.833187 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.833194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.833549 (+ 355us) server_negotiation.cc:407] Connection header received
0812 07:12:28.833654 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.833659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.833835 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.833988 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:28.834222 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.835343 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.836787 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.837363 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.837500 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.837727 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.839006 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.839126 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":118,"threads_started":1}
I20260812 07:12:28.841883 20456 tablet_service.cc:1467] Processing CreateTablet for tablet e52b1705b6c041e391a843c243fbea46 (DEFAULT_TABLE table=test-table [id=480b45a8969a40548118a48252084fc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:28.841899 20635 tablet_service.cc:1467] Processing CreateTablet for tablet e52b1705b6c041e391a843c243fbea46 (DEFAULT_TABLE table=test-table [id=480b45a8969a40548118a48252084fc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:28.841902 20795 tablet_service.cc:1467] Processing CreateTablet for tablet e52b1705b6c041e391a843c243fbea46 (DEFAULT_TABLE table=test-table [id=480b45a8969a40548118a48252084fc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:28.843153 20795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e52b1705b6c041e391a843c243fbea46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:28.843151 20635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e52b1705b6c041e391a843c243fbea46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:28.843152 20456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e52b1705b6c041e391a843c243fbea46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:28.850593 20894 tablet_bootstrap.cc:492] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf: Bootstrap starting.
I20260812 07:12:28.852254 20896 tablet_bootstrap.cc:492] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e: Bootstrap starting.
I20260812 07:12:28.854072 20894 tablet_bootstrap.cc:654] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:28.854460 20896 tablet_bootstrap.cc:654] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:28.854975 20895 tablet_bootstrap.cc:492] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210: Bootstrap starting.
I20260812 07:12:28.855296 20894 log.cc:824] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:28.856683 20896 log.cc:824] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:28.857390 20895 tablet_bootstrap.cc:654] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:28.858255 20895 log.cc:824] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:28.858843 20894 tablet_bootstrap.cc:492] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf: No bootstrap required, opened a new log
I20260812 07:12:28.859097 20894 ts_tablet_manager.cc:1397] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:12:28.859390 20896 tablet_bootstrap.cc:492] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e: No bootstrap required, opened a new log
I20260812 07:12:28.859689 20896 ts_tablet_manager.cc:1397] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:12:28.860718 20895 tablet_bootstrap.cc:492] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210: No bootstrap required, opened a new log
I20260812 07:12:28.860915 20895 ts_tablet_manager.cc:1397] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260812 07:12:28.864042 20896 raft_consensus.cc:348] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 } } peers { permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 } } peers { permanent_uuid: "d5522f334b014ece971e3a4a5a7826cf" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34989 } }
I20260812 07:12:28.864379 20896 raft_consensus.cc:374] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:28.864470 20896 raft_consensus.cc:729] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e12a5db66593442090e0675b1b4a640e, State: Initialized, Role: FOLLOWER
I20260812 07:12:28.864321 20894 raft_consensus.cc:348] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 } } peers { permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 } } peers { permanent_uuid: "d5522f334b014ece971e3a4a5a7826cf" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34989 } }
I20260812 07:12:28.864594 20894 raft_consensus.cc:374] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:28.864661 20894 raft_consensus.cc:729] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5522f334b014ece971e3a4a5a7826cf, State: Initialized, Role: FOLLOWER
I20260812 07:12:28.865108 20896 consensus_queue.cc:260] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 } } peers { permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 } } peers { permanent_uuid: "d5522f334b014ece971e3a4a5a7826cf" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34989 } }
I20260812 07:12:28.865139 20894 consensus_queue.cc:260] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 } } peers { permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 } } peers { permanent_uuid: "d5522f334b014ece971e3a4a5a7826cf" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34989 } }
I20260812 07:12:28.865375 20895 raft_consensus.cc:348] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 } } peers { permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 } } peers { permanent_uuid: "d5522f334b014ece971e3a4a5a7826cf" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34989 } }
I20260812 07:12:28.865701 20895 raft_consensus.cc:374] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:28.865983 20711 heartbeater.cc:503] Master 127.31.156.126:35195 was elected leader, sending a full tablet report...
I20260812 07:12:28.866235 20560 heartbeater.cc:503] Master 127.31.156.126:35195 was elected leader, sending a full tablet report...
I20260812 07:12:28.866366 20894 ts_tablet_manager.cc:1428] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:12:28.866374 20896 ts_tablet_manager.cc:1428] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260812 07:12:28.866636 20895 raft_consensus.cc:729] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ad50cf546eb424990e3e9de8e021210, State: Initialized, Role: FOLLOWER
I20260812 07:12:28.867192 20895 consensus_queue.cc:260] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 } } peers { permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 } } peers { permanent_uuid: "d5522f334b014ece971e3a4a5a7826cf" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34989 } }
I20260812 07:12:28.868046 20871 heartbeater.cc:503] Master 127.31.156.126:35195 was elected leader, sending a full tablet report...
I20260812 07:12:28.868368 20895 ts_tablet_manager.cc:1428] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260812 07:12:28.894548 20901 raft_consensus.cc:482] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:28.894874 20901 raft_consensus.cc:504] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 } } peers { permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 } } peers { permanent_uuid: "d5522f334b014ece971e3a4a5a7826cf" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34989 } }
I20260812 07:12:28.896281 20901 leader_election.cc:288] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e12a5db66593442090e0675b1b4a640e (127.31.156.66:46301), 9ad50cf546eb424990e3e9de8e021210 (127.31.156.67:34993)
I20260812 07:12:28.899524 20905 raft_consensus.cc:482] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:28.899782 20905 raft_consensus.cc:504] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 } } peers { permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 } } peers { permanent_uuid: "d5522f334b014ece971e3a4a5a7826cf" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34989 } }
I20260812 07:12:28.901198 20905 leader_election.cc:288] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e12a5db66593442090e0675b1b4a640e (127.31.156.66:46301), d5522f334b014ece971e3a4a5a7826cf (127.31.156.65:34989)
I20260812 07:12:28.902197 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.896829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:57741 (local address 127.31.156.66:46301)
0812 07:12:28.897000 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.897006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.897411 (+ 405us) server_negotiation.cc:407] Connection header received
0812 07:12:28.897623 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.897629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.897697 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.897804 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:28.898181 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.898808 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.900461 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.901202 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.901369 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.901431 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.901923 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.902034 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:12:28.902643 20891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.897397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:56555 (local address 127.31.156.67:34993)
0812 07:12:28.897562 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.897566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.897725 (+ 159us) server_negotiation.cc:407] Connection header received
0812 07:12:28.897893 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.897896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.897946 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.898032 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:28.898334 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.898874 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.900607 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.901276 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.901390 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.901434 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.902436 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.902533 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:12:28.902698 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.896741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:46301 (local address 127.31.156.65:57741)
0812 07:12:28.897097 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:28.897113 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:28.897476 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:28.897821 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:28.897824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:28.897844 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:28.897997 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.898004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.898934 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.898938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.900298 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.900311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.901514 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.901518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.901734 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.901767 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:12:28.902368 (+ 601us) client_negotiation.cc:240] Negotiation successful
0812 07:12:28.902471 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":86,"thread_start_us":129,"threads_started":1}
I20260812 07:12:28.903267 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.897184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:34993 (local address 127.31.156.65:56555)
0812 07:12:28.897626 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:28.897638 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:28.897774 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:28.898044 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:28.898047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:28.898063 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:28.898215 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.898220 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.899001 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.899005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.900452 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.900464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.901519 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.901522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.902166 (+ 644us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.902208 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 07:12:28.903004 (+ 796us) client_negotiation.cc:240] Negotiation successful
0812 07:12:28.903078 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":580,"thread_start_us":141,"threads_started":1}
I20260812 07:12:28.903669 20658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e52b1705b6c041e391a843c243fbea46" candidate_uuid: "d5522f334b014ece971e3a4a5a7826cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e12a5db66593442090e0675b1b4a640e" is_pre_election: true
I20260812 07:12:28.904060 20658 raft_consensus.cc:2454] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5522f334b014ece971e3a4a5a7826cf in term 0.
I20260812 07:12:28.904767 20424 leader_election.cc:302] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5522f334b014ece971e3a4a5a7826cf, e12a5db66593442090e0675b1b4a640e; no voters:
I20260812 07:12:28.904922 20822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e52b1705b6c041e391a843c243fbea46" candidate_uuid: "d5522f334b014ece971e3a4a5a7826cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ad50cf546eb424990e3e9de8e021210" is_pre_election: true
I20260812 07:12:28.905160 20822 raft_consensus.cc:2454] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5522f334b014ece971e3a4a5a7826cf in term 0.
I20260812 07:12:28.905164 20901 raft_consensus.cc:2781] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:12:28.905256 20901 raft_consensus.cc:482] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:12:28.905301 20901 raft_consensus.cc:3037] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:28.906975 20901 raft_consensus.cc:504] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 } } peers { permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 } } peers { permanent_uuid: "d5522f334b014ece971e3a4a5a7826cf" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34989 } }
I20260812 07:12:28.907183 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.901922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:53519 (local address 127.31.156.65:34989)
0812 07:12:28.902047 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.902051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.902300 (+ 249us) server_negotiation.cc:407] Connection header received
0812 07:12:28.902483 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.902486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.902536 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.902587 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:28.903310 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.904116 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.905905 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.906468 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.906615 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.906668 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.906948 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.907019 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:12:28.907289 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.901390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:46301 (local address 127.31.156.67:45495)
0812 07:12:28.901518 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:28.901535 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:28.901642 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:28.902764 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:28.902767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:28.902811 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:28.902940 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.902947 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.904287 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.904292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.905659 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.905685 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.906753 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.906761 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.906923 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.906956 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:12:28.907020 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 07:12:28.907103 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":3,"spinlock_wait_cycles":32384}
I20260812 07:12:28.907295 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.901681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34989 (local address 127.31.156.67:53519)
0812 07:12:28.902213 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:28.902228 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:28.902375 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:28.902764 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:28.902767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:28.902812 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:28.903174 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.903185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.904605 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.904608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.905752 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:28.905767 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.906622 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.906625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:28.906813 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.906834 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 07:12:28.906931 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:12:28.907012 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":222,"spinlock_wait_cycles":8064,"thread_start_us":151,"threads_started":1}
I20260812 07:12:28.907461 20901 leader_election.cc:288] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [CANDIDATE]: Term 1 election: Requested vote from peers e12a5db66593442090e0675b1b4a640e (127.31.156.66:46301), 9ad50cf546eb424990e3e9de8e021210 (127.31.156.67:34993)
I20260812 07:12:28.907933 20502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e52b1705b6c041e391a843c243fbea46" candidate_uuid: "9ad50cf546eb424990e3e9de8e021210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5522f334b014ece971e3a4a5a7826cf" is_pre_election: true
I20260812 07:12:28.907990 20658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e52b1705b6c041e391a843c243fbea46" candidate_uuid: "d5522f334b014ece971e3a4a5a7826cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e12a5db66593442090e0675b1b4a640e"
I20260812 07:12:28.907995 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.901820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:45495 (local address 127.31.156.66:46301)
0812 07:12:28.902206 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.902215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.902231 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:12:28.902494 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.902497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.902565 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.903202 (+ 637us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:28.903337 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.904107 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.905840 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.906542 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.907600 (+ 1058us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.907694 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.907746 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.907823 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":147,"threads_started":1}
I20260812 07:12:28.908129 20658 raft_consensus.cc:3037] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:28.908226 20502 raft_consensus.cc:2379] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9ad50cf546eb424990e3e9de8e021210 in current term 1: Already voted for candidate d5522f334b014ece971e3a4a5a7826cf in this term.
I20260812 07:12:28.908443 20657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e52b1705b6c041e391a843c243fbea46" candidate_uuid: "9ad50cf546eb424990e3e9de8e021210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e12a5db66593442090e0675b1b4a640e" is_pre_election: true
I20260812 07:12:28.909176 20732 leader_election.cc:302] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9ad50cf546eb424990e3e9de8e021210; no voters: d5522f334b014ece971e3a4a5a7826cf, e12a5db66593442090e0675b1b4a640e
I20260812 07:12:28.909277 20822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e52b1705b6c041e391a843c243fbea46" candidate_uuid: "d5522f334b014ece971e3a4a5a7826cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ad50cf546eb424990e3e9de8e021210"
I20260812 07:12:28.909404 20822 raft_consensus.cc:3037] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:28.909682 20658 raft_consensus.cc:2454] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5522f334b014ece971e3a4a5a7826cf in term 1.
I20260812 07:12:28.910095 20424 leader_election.cc:302] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5522f334b014ece971e3a4a5a7826cf, e12a5db66593442090e0675b1b4a640e; no voters:
I20260812 07:12:28.910391 20901 raft_consensus.cc:2781] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:28.910689 20901 raft_consensus.cc:686] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [term 1 LEADER]: Becoming Leader. State: Replica: d5522f334b014ece971e3a4a5a7826cf, State: Running, Role: LEADER
I20260812 07:12:28.911032 20822 raft_consensus.cc:2454] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5522f334b014ece971e3a4a5a7826cf in term 1.
I20260812 07:12:28.911173 20905 raft_consensus.cc:2726] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 07:12:28.911077 20901 consensus_queue.cc:237] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 } } peers { permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 } } peers { permanent_uuid: "d5522f334b014ece971e3a4a5a7826cf" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34989 } }
I20260812 07:12:28.915709 20330 catalog_manager.cc:5495] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf reported cstate change: term changed from 0 to 1, leader changed from <none> to d5522f334b014ece971e3a4a5a7826cf (127.31.156.65). New cstate: current_term: 1 leader_uuid: "d5522f334b014ece971e3a4a5a7826cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5522f334b014ece971e3a4a5a7826cf" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 34989 } health_report { overall_health: HEALTHY } } }
I20260812 07:12:28.940934 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.936634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39466 (local address 127.31.156.65:34989)
0812 07:12:28.936791 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:28.936794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:28.936850 (+ 56us) server_negotiation.cc:407] Connection header received
0812 07:12:28.936936 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:28.936939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:28.936997 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:28.937094 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:12:28.937458 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.938027 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.939010 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:28.939358 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:28.939452 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:28.939867 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:12:28.940422 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:12:28.940519 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:28.940737 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 07:12:28.940796 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:12:28.948282 20658 raft_consensus.cc:1261] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e [term 1 FOLLOWER]: Refusing update from remote peer d5522f334b014ece971e3a4a5a7826cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:28.948490 20822 raft_consensus.cc:1261] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 1 FOLLOWER]: Refusing update from remote peer d5522f334b014ece971e3a4a5a7826cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:28.949474 20901 consensus_queue.cc:1035] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e12a5db66593442090e0675b1b4a640e" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 46301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:28.949800 20911 consensus_queue.cc:1035] T e52b1705b6c041e391a843c243fbea46 P d5522f334b014ece971e3a4a5a7826cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ad50cf546eb424990e3e9de8e021210" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 34993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:28.973018 20333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:28.973213 20333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:28.976703 20333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d12459da2834192967f01717c81320a: Sending DeleteTablet for 3 replicas of tablet e52b1705b6c041e391a843c243fbea46
I20260812 07:12:28.977882 20635 tablet_service.cc:1514] Processing DeleteTablet for tablet e52b1705b6c041e391a843c243fbea46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:28 UTC) from {username='slave'} at 127.0.0.1:50950
I20260812 07:12:28.977991 20456 tablet_service.cc:1514] Processing DeleteTablet for tablet e52b1705b6c041e391a843c243fbea46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:28 UTC) from {username='slave'} at 127.0.0.1:39464
I20260812 07:12:28.978545 20925 tablet_replica.cc:330] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e: stopping tablet replica
I20260812 07:12:28.978781 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 20405
I20260812 07:12:28.978842 20795 tablet_service.cc:1514] Processing DeleteTablet for tablet e52b1705b6c041e391a843c243fbea46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:28 UTC) from {username='slave'} at 127.0.0.1:57828
I20260812 07:12:28.980511 20928 tablet_replica.cc:330] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210: stopping tablet replica
I20260812 07:12:28.980929 20925 raft_consensus.cc:2229] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:28.981045 20928 raft_consensus.cc:2229] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:28.981341 20925 raft_consensus.cc:2258] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:28.981478 20928 raft_consensus.cc:2258] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:28.984045 20925 ts_tablet_manager.cc:1905] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:28.984820 20928 ts_tablet_manager.cc:1905] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:28.987551 20925 ts_tablet_manager.cc:1918] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:28.987685 20925 log.cc:1196] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/e52b1705b6c041e391a843c243fbea46
I20260812 07:12:28.988085 20925 ts_tablet_manager.cc:1939] T e52b1705b6c041e391a843c243fbea46 P e12a5db66593442090e0675b1b4a640e: Deleting consensus metadata
I20260812 07:12:28.988220 20928 ts_tablet_manager.cc:1918] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:28.988343 20928 log.cc:1196] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/e52b1705b6c041e391a843c243fbea46
I20260812 07:12:28.988690 20928 ts_tablet_manager.cc:1939] T e52b1705b6c041e391a843c243fbea46 P 9ad50cf546eb424990e3e9de8e021210: Deleting consensus metadata
I20260812 07:12:28.989990 20315 catalog_manager.cc:4841] TS e12a5db66593442090e0675b1b4a640e (127.31.156.66:46301): tablet e52b1705b6c041e391a843c243fbea46 (table test-table [id=480b45a8969a40548118a48252084fc7]) successfully deleted
I20260812 07:12:28.990290 20315 catalog_manager.cc:4841] TS 9ad50cf546eb424990e3e9de8e021210 (127.31.156.67:34993): tablet e52b1705b6c041e391a843c243fbea46 (table test-table [id=480b45a8969a40548118a48252084fc7]) successfully deleted
W20260812 07:12:28.991935 20315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:34989 (error 108)
I20260812 07:12:28.992700 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 20563
I20260812 07:12:28.997238 20892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:28.992903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:34989 (local address 127.0.0.1:39476)
0812 07:12:28.996956 (+ 4053us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:28.997086 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:34989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3893}
W20260812 07:12:28.997670 20315 catalog_manager.cc:4584] TS d5522f334b014ece971e3a4a5a7826cf (127.31.156.65:34989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e52b1705b6c041e391a843c243fbea46: Network error: Client connection negotiation failed: client connection to 127.31.156.65:34989: connect: Connection refused (error 111)
I20260812 07:12:29.008852 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 20714
I20260812 07:12:29.020584 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 20299
2026-08-12T07:12:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1187 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:29 dist-test-slave-csg5 krb5kdc[20932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:29 dist-test-slave-csg5 krb5kdc[20932](info): set up 2 sockets
Aug 12 07:12:29 dist-test-slave-csg5 krb5kdc[20932](info): commencing operation
krb5kdc: starting...
W20260812 07:12:30.208531 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:30 dist-test-slave-csg5 krb5kdc[20932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:12:30.389467 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:33729
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:30.514897 20950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:30.515259 20950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:30.515345 20950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:30.519704 20950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:30.519817 20950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:30.519855 20950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:30.519893 20950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:30.519934 20950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:30.525801 20950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:33729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:33729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:30.527070 20950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:30.528290 20950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:30.538614 20957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:30.538713 20956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:30.538833 20950 server_base.cc:1034] running on GCE node
W20260812 07:12:30.538618 20960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:30.539731 20950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:30.541040 20950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:30.542380 20950 hybrid_clock.cc:648] HybridClock initialized: now 1786518750542334 us; error 55 us; skew 500 ppm
Aug 12 07:12:30 dist-test-slave-csg5 krb5kdc[20932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518750, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:30.551057 20950 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:12:30.552244 20950 webserver.cc:460] Webserver started at http://127.31.156.126:45063/ using document root <none> and password file <none>
I20260812 07:12:30.553122 20950 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:30.553241 20950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:30.553450 20950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:30.555924 20950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "64e1c8ad16fd45728c460f07a73b376b"
format_stamp: "Formatted at 2026-08-12 07:12:30 on dist-test-slave-csg5"
server_key: "0567812355097d63576abb239117c902"
server_key_iv: "9488c77fb9dc5a9478bcbcde04acb511"
server_key_version: "encryptionkey@0"
I20260812 07:12:30.556494 20950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "64e1c8ad16fd45728c460f07a73b376b"
format_stamp: "Formatted at 2026-08-12 07:12:30 on dist-test-slave-csg5"
server_key: "0567812355097d63576abb239117c902"
server_key_iv: "9488c77fb9dc5a9478bcbcde04acb511"
server_key_version: "encryptionkey@0"
I20260812 07:12:30.560988 20950 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 07:12:30.564093 20967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:30.565824 20950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:12:30.566004 20950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "64e1c8ad16fd45728c460f07a73b376b"
format_stamp: "Formatted at 2026-08-12 07:12:30 on dist-test-slave-csg5"
server_key: "0567812355097d63576abb239117c902"
server_key_iv: "9488c77fb9dc5a9478bcbcde04acb511"
server_key_version: "encryptionkey@0"
I20260812 07:12:30.566139 20950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:30.581243 20950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:30.585382 20950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:30.585597 20950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:30.594172 20950 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:33729
I20260812 07:12:30.594269 21021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:33729 every 8 connection(s)
I20260812 07:12:30.595355 20950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:30.598879 21022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:30.599157 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 20950
I20260812 07:12:30.599323 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:30.599700 32369 external_mini_cluster.cc:1428] Setting key 2f4dab097f2357497d409109bb3de328
I20260812 07:12:30.607811 21022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b: Bootstrap starting.
Aug 12 07:12:30 dist-test-slave-csg5 krb5kdc[20932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:12:30.611115 21022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:30.612010 21022 log.cc:824] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:30.615024 21022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b: No bootstrap required, opened a new log
I20260812 07:12:30.617429 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:30.601470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55806 (local address 127.31.156.126:33729)
0812 07:12:30.602075 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:30.602093 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:30.602136 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:12:30.603015 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:30.603036 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:30.603456 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:30.603943 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:30.604983 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:30.606171 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:30.607318 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:30.607933 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:30.608148 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:30.611526 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:30.611532 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:30.611549 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:30.611585 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:30.613695 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:30.614280 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:30.614286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:30.614291 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:30.614499 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:30.614881 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:30.614884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:30.614885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:30.615663 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:30.615903 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:30.616220 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 07:12:30.616602 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":187,"threads_started":1}
I20260812 07:12:30.619328 21022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e1c8ad16fd45728c460f07a73b376b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33729 } }
I20260812 07:12:30.619598 21022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:30.619709 21022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e1c8ad16fd45728c460f07a73b376b, State: Initialized, Role: FOLLOWER
I20260812 07:12:30.620177 21022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e1c8ad16fd45728c460f07a73b376b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33729 } }
I20260812 07:12:30.620357 21022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:30.620448 21022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:30.620574 21022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:30.621713 21022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e1c8ad16fd45728c460f07a73b376b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33729 } }
I20260812 07:12:30.622099 21022 leader_election.cc:302] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64e1c8ad16fd45728c460f07a73b376b; no voters:
I20260812 07:12:30.622467 21022 leader_election.cc:288] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:30.622808 21029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:30.623073 21029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [term 1 LEADER]: Becoming Leader. State: Replica: 64e1c8ad16fd45728c460f07a73b376b, State: Running, Role: LEADER
I20260812 07:12:30.623716 21029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e1c8ad16fd45728c460f07a73b376b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33729 } }
I20260812 07:12:30.624082 21022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:30.625890 21030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64e1c8ad16fd45728c460f07a73b376b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e1c8ad16fd45728c460f07a73b376b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33729 } } }
I20260812 07:12:30.626063 21030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:30.626073 21031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e1c8ad16fd45728c460f07a73b376b. Latest consensus state: current_term: 1 leader_uuid: "64e1c8ad16fd45728c460f07a73b376b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e1c8ad16fd45728c460f07a73b376b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 33729 } } }
I20260812 07:12:30.626164 21031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b [sys.catalog]: This master's current role is: LEADER
W20260812 07:12:30.631323 21047 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:12:30.631428 21047 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:12:30.631903 21046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:30.633035 21046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:30.640580 21046 catalog_manager.cc:1347] Generated new cluster ID: 12258bf946e74b889234283902c0e5da
I20260812 07:12:30.640704 21046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:30.662497 21046 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:30.664233 21046 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:30.674774 21046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b: Generated new TSK 0
I20260812 07:12:30.675949 21046 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:12:30.716187 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33729
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:30.842054 21054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:30.842367 21054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:30.842413 21054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:30.846709 21054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:30.846796 21054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:30.846860 21054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:12:30.852419 21054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:30.853590 21054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:30.854759 21054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:30.866748 21063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:30.866869 21054 server_base.cc:1034] running on GCE node
W20260812 07:12:30.866658 21061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:30.866648 21059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:30.867653 21054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:30.868484 21054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:30.869767 21054 hybrid_clock.cc:648] HybridClock initialized: now 1786518750869754 us; error 35 us; skew 500 ppm
Aug 12 07:12:30 dist-test-slave-csg5 krb5kdc[20932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518750, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:30.873395 21054 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:12:30.874506 21054 webserver.cc:460] Webserver started at http://127.31.156.65:36671/ using document root <none> and password file <none>
I20260812 07:12:30.875242 21054 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:30.875380 21054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:30.875782 21054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:30.878577 21054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "939f8ef107804ff18436c3374f446efc"
format_stamp: "Formatted at 2026-08-12 07:12:30 on dist-test-slave-csg5"
server_key: "b3d979e1a98931228df5b98e511fb41e"
server_key_iv: "d34be114b4c7b0b1d0b7d86e7704c761"
server_key_version: "encryptionkey@0"
I20260812 07:12:30.879377 21054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "939f8ef107804ff18436c3374f446efc"
format_stamp: "Formatted at 2026-08-12 07:12:30 on dist-test-slave-csg5"
server_key: "b3d979e1a98931228df5b98e511fb41e"
server_key_iv: "d34be114b4c7b0b1d0b7d86e7704c761"
server_key_version: "encryptionkey@0"
I20260812 07:12:30.884202 21054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:30.887538 21071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:30.889145 21054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:12:30.889277 21054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "939f8ef107804ff18436c3374f446efc"
format_stamp: "Formatted at 2026-08-12 07:12:30 on dist-test-slave-csg5"
server_key: "b3d979e1a98931228df5b98e511fb41e"
server_key_iv: "d34be114b4c7b0b1d0b7d86e7704c761"
server_key_version: "encryptionkey@0"
I20260812 07:12:30.889452 21054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:30.908146 21054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:30.912130 21054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:30.912369 21054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:30.913061 21054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:30.913997 21054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:30.914048 21054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:30.914090 21054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:30.914151 21054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:30.927175 21054 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:42503
I20260812 07:12:30.927233 21197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:42503 every 8 connection(s)
I20260812 07:12:30.928208 21054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:30.935537 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 21054
I20260812 07:12:30.935671 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:12:30.936012 32369 external_mini_cluster.cc:1428] Setting key 99f353cb83a31b08a7df93a47b359e34
Aug 12 07:12:30 dist-test-slave-csg5 krb5kdc[20932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518750, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:12:30.949481 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:30.930560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:41637 (local address 127.31.156.126:33729)
0812 07:12:30.930771 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:30.930776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:30.931811 (+ 1035us) server_negotiation.cc:407] Connection header received
0812 07:12:30.933340 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:30.933344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:30.933405 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:30.933716 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:30.935330 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:30.936191 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:30.938205 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:30.938544 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:30.938647 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:30.942830 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:30.942838 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:30.942844 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:30.942887 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:30.945330 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:30.946489 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:30.946496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:30.946499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:30.946628 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:30.947201 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:30.947207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:30.947210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:30.947779 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:30.947913 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:30.949021 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0812 07:12:30.949223 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:12:30.950383 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:30.931010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33729 (local address 127.31.156.65:41637)
0812 07:12:30.931563 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:30.931629 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:30.933032 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:30.933966 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:30.933975 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:30.934437 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:30.935108 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:30.935125 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:30.936450 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:30.936454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:30.937999 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:30.938014 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:30.938805 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:30.938813 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:30.939121 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:30.940097 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:30.940132 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:30.942504 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:30.945626 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:30.945638 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:30.945665 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:30.946085 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:30.946852 (+ 767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:30.946855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:30.946856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:30.947061 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:30.947968 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:30.947981 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:30.948604 (+ 623us) client_negotiation.cc:769] Sending connection context
0812 07:12:30.948888 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 07:12:30.949306 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":140,"threads_started":1}
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
I20260812 07:12:30.952183 21198 heartbeater.cc:347] Connected to a master server at 127.31.156.126:33729
I20260812 07:12:30.952594 21198 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:30.953374 21198 heartbeater.cc:511] Master 127.31.156.126:33729 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
I20260812 07:12:30.955861 20985 ts_manager.cc:194] Registered new tserver with Master: 939f8ef107804ff18436c3374f446efc (127.31.156.65:42503)
I20260812 07:12:30.957664 20985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:41637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:12:30.980459 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33729
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:31.120982 21204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:31.121377 21204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:31.121492 21204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:31.126957 21204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:31.127068 21204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:31.127177 21204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:12:31.133720 21204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:31.135265 21204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:31.136498 21204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:31.149461 21214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:31.149487 21212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:31.149461 21211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:31.149945 21204 server_base.cc:1034] running on GCE node
I20260812 07:12:31.150619 21204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:31.151486 21204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:31.152709 21204 hybrid_clock.cc:648] HybridClock initialized: now 1786518751152687 us; error 39 us; skew 500 ppm
Aug 12 07:12:31 dist-test-slave-csg5 krb5kdc[20932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518751, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:31.156461 21204 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:12:31.157682 21204 webserver.cc:460] Webserver started at http://127.31.156.66:33015/ using document root <none> and password file <none>
I20260812 07:12:31.158315 21204 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:31.158384 21204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:31.158617 21204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:31.161023 21204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "5fa134a063644e579702cde50edb2da5"
format_stamp: "Formatted at 2026-08-12 07:12:31 on dist-test-slave-csg5"
server_key: "c39dd6c6e4f78b7738fe5262429185dc"
server_key_iv: "a591ca0fb166ebe348e3c4fe1edad572"
server_key_version: "encryptionkey@0"
I20260812 07:12:31.161703 21204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "5fa134a063644e579702cde50edb2da5"
format_stamp: "Formatted at 2026-08-12 07:12:31 on dist-test-slave-csg5"
server_key: "c39dd6c6e4f78b7738fe5262429185dc"
server_key_iv: "a591ca0fb166ebe348e3c4fe1edad572"
server_key_version: "encryptionkey@0"
I20260812 07:12:31.166312 21204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 07:12:31.169663 21222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:31.171535 21204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:12:31.171733 21204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "5fa134a063644e579702cde50edb2da5"
format_stamp: "Formatted at 2026-08-12 07:12:31 on dist-test-slave-csg5"
server_key: "c39dd6c6e4f78b7738fe5262429185dc"
server_key_iv: "a591ca0fb166ebe348e3c4fe1edad572"
server_key_version: "encryptionkey@0"
I20260812 07:12:31.171921 21204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:31.188743 21204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:31.193010 21204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:31.193267 21204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:31.194087 21204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:31.195233 21204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:31.195327 21204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:31.195413 21204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:31.195454 21204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:31.206627 21204 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:43805
I20260812 07:12:31.206769 21356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:43805 every 8 connection(s)
I20260812 07:12:31.207849 21204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:12:31.210484 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 21204
I20260812 07:12:31.210704 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:12:31.211107 32369 external_mini_cluster.cc:1428] Setting key e9b7fceccedda15d12d4784868bbaff6
Aug 12 07:12:31 dist-test-slave-csg5 krb5kdc[20932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518751, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:12:31.227192 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.210156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:39757 (local address 127.31.156.126:33729)
0812 07:12:31.210384 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:31.210389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:31.211576 (+ 1187us) server_negotiation.cc:407] Connection header received
0812 07:12:31.213402 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:31.213409 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:31.213478 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:31.213585 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:31.215279 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.215834 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.218096 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.218437 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.218550 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.222434 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:31.222438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:31.222442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:31.222483 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:31.224467 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:31.225252 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:31.225256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:31.225258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:31.225358 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:31.225797 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:31.225800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:31.225802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:31.226089 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:31.226203 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:31.226856 (+ 653us) server_negotiation.cc:299] Negotiation successful
0812 07:12:31.227013 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260812 07:12:31.228272 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.210582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33729 (local address 127.31.156.66:39757)
0812 07:12:31.211341 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:31.211420 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:31.213099 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:31.213828 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:31.213837 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:31.214311 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:31.215052 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.215082 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.215997 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.216005 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.217890 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.217910 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.218689 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.218697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.219021 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.219950 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:31.219978 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:31.222153 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:31.224622 (+ 2469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:31.224630 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:31.224656 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:31.225123 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:31.225464 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:31.225485 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:31.225489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:31.225674 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:31.226205 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:31.226216 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:31.226535 (+ 319us) client_negotiation.cc:769] Sending connection context
0812 07:12:31.226795 (+ 260us) client_negotiation.cc:240] Negotiation successful
0812 07:12:31.227190 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":163,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
I20260812 07:12:31.229949 21357 heartbeater.cc:347] Connected to a master server at 127.31.156.126:33729
I20260812 07:12:31.230368 21357 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:31.231179 21357 heartbeater.cc:511] Master 127.31.156.126:33729 requested a full tablet report, sending...
I20260812 07:12:31.232949 20985 ts_manager.cc:194] Registered new tserver with Master: 5fa134a063644e579702cde50edb2da5 (127.31.156.66:43805)
I20260812 07:12:31.233736 20985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:39757
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:12:31.253386 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33729
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:31.392292 21365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:31.392709 21365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:31.392791 21365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:31.397356 21365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:31.397485 21365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:31.397594 21365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:12:31.403650 21365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:33729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:31.404969 21365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:31.406175 21365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:31.418521 21372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:31.418557 21371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:31.418521 21374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:31.418927 21365 server_base.cc:1034] running on GCE node
I20260812 07:12:31.419513 21365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:31.420305 21365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:31.421512 21365 hybrid_clock.cc:648] HybridClock initialized: now 1786518751421461 us; error 61 us; skew 500 ppm
Aug 12 07:12:31 dist-test-slave-csg5 krb5kdc[20932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518751, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:31.425606 21365 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:12:31.426839 21365 webserver.cc:460] Webserver started at http://127.31.156.67:40061/ using document root <none> and password file <none>
I20260812 07:12:31.427457 21365 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:31.427527 21365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:31.427728 21365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:31.431123 21365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "d0899e5ef5d8434d906a00e06474c587"
format_stamp: "Formatted at 2026-08-12 07:12:31 on dist-test-slave-csg5"
server_key: "1308ec15d145db95cdf7297b6e341248"
server_key_iv: "9deddfca9759b8463594feaae83248bb"
server_key_version: "encryptionkey@0"
I20260812 07:12:31.431947 21365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "d0899e5ef5d8434d906a00e06474c587"
format_stamp: "Formatted at 2026-08-12 07:12:31 on dist-test-slave-csg5"
server_key: "1308ec15d145db95cdf7297b6e341248"
server_key_iv: "9deddfca9759b8463594feaae83248bb"
server_key_version: "encryptionkey@0"
I20260812 07:12:31.436875 21365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:12:31.440094 21383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:31.441813 21365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:12:31.442011 21365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "d0899e5ef5d8434d906a00e06474c587"
format_stamp: "Formatted at 2026-08-12 07:12:31 on dist-test-slave-csg5"
server_key: "1308ec15d145db95cdf7297b6e341248"
server_key_iv: "9deddfca9759b8463594feaae83248bb"
server_key_version: "encryptionkey@0"
I20260812 07:12:31.442174 21365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:31.481216 21365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:31.486086 21365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:31.486382 21365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:31.487176 21365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:31.488570 21365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:31.488673 21365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:31.488758 21365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:31.488801 21365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:31.500006 21365 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:40505
I20260812 07:12:31.500027 21518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:40505 every 8 connection(s)
I20260812 07:12:31.501129 21365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:31.510223 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 21365
I20260812 07:12:31.510373 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:12:31.510819 32369 external_mini_cluster.cc:1428] Setting key 3922c63ffb6ff1bfe7dd0351441e3862
Aug 12 07:12:31 dist-test-slave-csg5 krb5kdc[20932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518751, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:12:31.518505 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.503590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:35723 (local address 127.31.156.126:33729)
0812 07:12:31.503825 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:31.503830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:31.504977 (+ 1147us) server_negotiation.cc:407] Connection header received
0812 07:12:31.506628 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:31.506632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:31.506738 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:31.506870 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:31.508387 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.509005 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.510450 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.510810 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.510966 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.514458 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:31.514461 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:31.514464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:31.514493 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:31.516290 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:31.516898 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:31.516901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:31.516902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:31.516987 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:31.517385 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:31.517387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:31.517388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:31.517587 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:31.517661 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:31.518252 (+ 591us) server_negotiation.cc:299] Negotiation successful
0812 07:12:31.518369 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260812 07:12:31.519853 21521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.504070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:33729 (local address 127.31.156.67:35723)
0812 07:12:31.504755 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:31.504813 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:31.506328 (+ 1515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:31.507101 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:31.507111 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:31.507573 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:31.508190 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.508207 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.509166 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.509170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.510224 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.510235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.511053 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.511057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.511337 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.512185 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:31.512216 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:31.514220 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:31.516453 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:31.516460 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:31.516477 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:31.516789 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:31.517084 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:31.517086 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:31.517088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:31.517258 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:31.517690 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:31.517696 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:31.517988 (+ 292us) client_negotiation.cc:769] Sending connection context
0812 07:12:31.518171 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 07:12:31.518533 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":159,"threads_started":1}
I20260812 07:12:31.521252 21519 heartbeater.cc:347] Connected to a master server at 127.31.156.126:33729
I20260812 07:12:31.521549 21519 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:31.522154 21519 heartbeater.cc:511] Master 127.31.156.126:33729 requested a full tablet report, sending...
I20260812 07:12:31.523418 20985 ts_manager.cc:194] Registered new tserver with Master: d0899e5ef5d8434d906a00e06474c587 (127.31.156.67:40505)
I20260812 07:12:31.524228 20985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:35723
I20260812 07:12:31.526182 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:12:31.537995 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.528396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55822 (local address 127.31.156.126:33729)
0812 07:12:31.528554 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:31.528558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:31.528771 (+ 213us) server_negotiation.cc:407] Connection header received
0812 07:12:31.528854 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:31.528857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:31.528899 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:31.528974 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:31.529827 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.530434 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.531525 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.531792 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.531894 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.533896 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:31.533900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:31.533902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:31.533954 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:31.535878 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:31.536561 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:31.536564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:31.536565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:31.536654 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:31.537039 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:31.537041 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:31.537042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:31.537282 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:31.537453 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:31.537752 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 07:12:31.537862 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:12:31.541502 20985 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:12:31.544875 20985 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:12:31.562008 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.556783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37134 (local address 127.31.156.66:43805)
0812 07:12:31.557332 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:31.557344 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:31.557364 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:12:31.557435 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:31.557439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:31.557619 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:31.557768 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:31.558026 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.559166 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.560401 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.561115 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.561231 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.561399 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:31.561668 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 07:12:31.561788 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":131,"threads_started":1}
I20260812 07:12:31.562000 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.556595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:43805 (local address 127.0.0.1:37134)
0812 07:12:31.556967 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:31.556980 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:31.557066 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:31.557775 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:31.557778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:31.557800 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:31.557887 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.557893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.559291 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.559298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.560253 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.560264 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.561262 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.561266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.561391 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.561440 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 07:12:31.561609 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 07:12:31.561777 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":48,"thread_start_us":105,"threads_started":1}
I20260812 07:12:31.562347 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.555384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:40505 (local address 127.0.0.1:43794)
0812 07:12:31.556150 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:31.556207 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:31.556338 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:31.557230 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:31.557237 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:31.557263 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:31.557426 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.557434 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.559073 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.559076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.560750 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.560756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.561923 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.561926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.562028 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.562060 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:12:31.562120 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 07:12:31.562219 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":134,"thread_start_us":110,"threads_started":1}
I20260812 07:12:31.562641 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.555506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:42503 (local address 127.0.0.1:51424)
0812 07:12:31.556189 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:31.556207 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:31.556339 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:31.557402 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:31.557405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:31.557422 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:31.557515 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.557523 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.559014 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.559018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.560742 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.560755 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.562187 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.562192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.562285 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.562310 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:12:31.562391 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 07:12:31.562479 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":81,"spinlock_wait_cycles":1792,"thread_start_us":142,"threads_started":1}
I20260812 07:12:31.563059 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.556147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43794 (local address 127.31.156.67:40505)
0812 07:12:31.556656 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:31.556675 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:31.556705 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:12:31.556791 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:31.556800 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:31.557037 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:31.557196 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:31.557575 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.558940 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.560894 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.561731 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.562545 (+ 814us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.562731 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:31.562789 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 07:12:31.562910 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":191,"threads_started":1}
I20260812 07:12:31.564044 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.555655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.31.156.65:42503)
0812 07:12:31.556674 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:31.556684 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:31.556711 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:12:31.556797 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:31.556802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:31.557034 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:31.557238 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:31.557626 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.558861 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.560912 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.561832 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.561945 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.562102 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:31.562974 (+ 872us) server_negotiation.cc:299] Negotiation successful
0812 07:12:31.563119 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":889,"thread_start_us":81,"threads_started":1}
I20260812 07:12:31.566145 21443 tablet_service.cc:1467] Processing CreateTablet for tablet 2d176a97a76b4719bc7c364739c24ef6 (DEFAULT_TABLE table=test-table [id=a4a233cc0e34420c92e9ae59278246a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:31.566148 21279 tablet_service.cc:1467] Processing CreateTablet for tablet 2d176a97a76b4719bc7c364739c24ef6 (DEFAULT_TABLE table=test-table [id=a4a233cc0e34420c92e9ae59278246a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:31.567454 21279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d176a97a76b4719bc7c364739c24ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:31.567456 21443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d176a97a76b4719bc7c364739c24ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:31.567373 21127 tablet_service.cc:1467] Processing CreateTablet for tablet 2d176a97a76b4719bc7c364739c24ef6 (DEFAULT_TABLE table=test-table [id=a4a233cc0e34420c92e9ae59278246a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:31.568455 21127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d176a97a76b4719bc7c364739c24ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:31.575191 21540 tablet_bootstrap.cc:492] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: Bootstrap starting.
I20260812 07:12:31.576519 21542 tablet_bootstrap.cc:492] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc: Bootstrap starting.
I20260812 07:12:31.577889 21540 tablet_bootstrap.cc:654] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:31.578579 21542 tablet_bootstrap.cc:654] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:31.579653 21542 log.cc:824] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:31.579761 21540 log.cc:824] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:31.580662 21541 tablet_bootstrap.cc:492] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: Bootstrap starting.
I20260812 07:12:31.582180 21540 tablet_bootstrap.cc:492] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: No bootstrap required, opened a new log
I20260812 07:12:31.582441 21540 ts_tablet_manager.cc:1397] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:12:31.583602 21541 tablet_bootstrap.cc:654] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:31.584497 21542 tablet_bootstrap.cc:492] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc: No bootstrap required, opened a new log
I20260812 07:12:31.584735 21542 ts_tablet_manager.cc:1397] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 07:12:31.585208 21541 log.cc:824] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:31.586009 21540 raft_consensus.cc:348] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0899e5ef5d8434d906a00e06474c587" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40505 } } peers { permanent_uuid: "939f8ef107804ff18436c3374f446efc" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42503 } } peers { permanent_uuid: "5fa134a063644e579702cde50edb2da5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43805 } }
I20260812 07:12:31.586323 21540 raft_consensus.cc:374] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:31.586390 21540 raft_consensus.cc:729] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0899e5ef5d8434d906a00e06474c587, State: Initialized, Role: FOLLOWER
I20260812 07:12:31.586993 21540 consensus_queue.cc:260] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0899e5ef5d8434d906a00e06474c587" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40505 } } peers { permanent_uuid: "939f8ef107804ff18436c3374f446efc" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42503 } } peers { permanent_uuid: "5fa134a063644e579702cde50edb2da5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43805 } }
I20260812 07:12:31.588078 21541 tablet_bootstrap.cc:492] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: No bootstrap required, opened a new log
I20260812 07:12:31.588110 21540 ts_tablet_manager.cc:1428] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260812 07:12:31.588310 21541 ts_tablet_manager.cc:1397] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 07:12:31.588507 21519 heartbeater.cc:503] Master 127.31.156.126:33729 was elected leader, sending a full tablet report...
I20260812 07:12:31.590039 21542 raft_consensus.cc:348] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0899e5ef5d8434d906a00e06474c587" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40505 } } peers { permanent_uuid: "939f8ef107804ff18436c3374f446efc" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42503 } } peers { permanent_uuid: "5fa134a063644e579702cde50edb2da5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43805 } }
I20260812 07:12:31.590395 21542 raft_consensus.cc:374] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:31.590476 21542 raft_consensus.cc:729] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939f8ef107804ff18436c3374f446efc, State: Initialized, Role: FOLLOWER
I20260812 07:12:31.591050 21542 consensus_queue.cc:260] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0899e5ef5d8434d906a00e06474c587" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40505 } } peers { permanent_uuid: "939f8ef107804ff18436c3374f446efc" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42503 } } peers { permanent_uuid: "5fa134a063644e579702cde50edb2da5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43805 } }
I20260812 07:12:31.592265 21198 heartbeater.cc:503] Master 127.31.156.126:33729 was elected leader, sending a full tablet report...
I20260812 07:12:31.592270 21542 ts_tablet_manager.cc:1428] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc: Time spent starting tablet: real 0.007s user 0.009s sys 0.000s
I20260812 07:12:31.592789 21541 raft_consensus.cc:348] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0899e5ef5d8434d906a00e06474c587" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40505 } } peers { permanent_uuid: "939f8ef107804ff18436c3374f446efc" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42503 } } peers { permanent_uuid: "5fa134a063644e579702cde50edb2da5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43805 } }
I20260812 07:12:31.593133 21541 raft_consensus.cc:374] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:31.593227 21541 raft_consensus.cc:729] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa134a063644e579702cde50edb2da5, State: Initialized, Role: FOLLOWER
I20260812 07:12:31.593770 21541 consensus_queue.cc:260] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0899e5ef5d8434d906a00e06474c587" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40505 } } peers { permanent_uuid: "939f8ef107804ff18436c3374f446efc" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42503 } } peers { permanent_uuid: "5fa134a063644e579702cde50edb2da5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43805 } }
I20260812 07:12:31.594971 21357 heartbeater.cc:503] Master 127.31.156.126:33729 was elected leader, sending a full tablet report...
I20260812 07:12:31.595036 21541 ts_tablet_manager.cc:1428] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260812 07:12:31.680277 21199 tablet.cc:2369] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:12:31.709628 21359 tablet.cc:2369] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:12:31.752408 21520 tablet.cc:2369] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:12:31.947841 21546 raft_consensus.cc:482] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:31.948189 21546 raft_consensus.cc:504] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0899e5ef5d8434d906a00e06474c587" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40505 } } peers { permanent_uuid: "939f8ef107804ff18436c3374f446efc" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42503 } } peers { permanent_uuid: "5fa134a063644e579702cde50edb2da5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43805 } }
I20260812 07:12:31.950461 21546 leader_election.cc:288] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939f8ef107804ff18436c3374f446efc (127.31.156.65:42503), 5fa134a063644e579702cde50edb2da5 (127.31.156.66:43805)
I20260812 07:12:31.957238 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.951122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:41709 (local address 127.31.156.66:43805)
0812 07:12:31.951619 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:31.951627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:31.951671 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:12:31.951776 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:31.951783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:31.951854 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:31.952004 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:31.952507 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.953301 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.955386 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.956095 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.956344 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.956458 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:31.956790 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 07:12:31.956956 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351}
I20260812 07:12:31.957253 21521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.950996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:43805 (local address 127.31.156.67:41709)
0812 07:12:31.951280 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:31.951300 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:31.951526 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:31.952023 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:31.952027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:31.952048 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:31.952292 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.952310 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.953588 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.953597 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.955153 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.955167 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.956342 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.956359 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.956559 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.956594 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 07:12:31.956764 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 07:12:31.956976 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":1}
I20260812 07:12:31.957710 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.951101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:42503 (local address 127.31.156.67:51277)
0812 07:12:31.951614 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:31.951634 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:31.951915 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:31.952377 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:31.952381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:31.952409 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:31.952664 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.952682 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.954044 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.954049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.955542 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.955564 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.956725 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.956731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.956977 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.957011 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 07:12:31.957210 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 07:12:31.957366 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":4,"thread_start_us":140,"threads_started":1}
I20260812 07:12:31.958271 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.951302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:51277 (local address 127.31.156.65:42503)
0812 07:12:31.951612 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:31.951618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:31.951820 (+ 202us) server_negotiation.cc:407] Connection header received
0812 07:12:31.952087 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:31.952090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:31.952183 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:31.952373 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:31.952866 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.953697 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.955812 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.956450 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.956737 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.956883 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:31.957242 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 07:12:31.958053 (+ 811us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260812 07:12:31.958626 21303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d176a97a76b4719bc7c364739c24ef6" candidate_uuid: "d0899e5ef5d8434d906a00e06474c587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa134a063644e579702cde50edb2da5" is_pre_election: true
I20260812 07:12:31.959272 21303 raft_consensus.cc:2454] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0899e5ef5d8434d906a00e06474c587 in term 0.
I20260812 07:12:31.959412 21147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d176a97a76b4719bc7c364739c24ef6" candidate_uuid: "d0899e5ef5d8434d906a00e06474c587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939f8ef107804ff18436c3374f446efc" is_pre_election: true
I20260812 07:12:31.959971 21147 raft_consensus.cc:2454] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0899e5ef5d8434d906a00e06474c587 in term 0.
I20260812 07:12:31.960320 21385 leader_election.cc:302] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa134a063644e579702cde50edb2da5, d0899e5ef5d8434d906a00e06474c587; no voters:
I20260812 07:12:31.960889 21546 raft_consensus.cc:2781] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:12:31.961105 21546 raft_consensus.cc:482] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:12:31.961268 21546 raft_consensus.cc:3037] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:31.963485 21546 raft_consensus.cc:504] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0899e5ef5d8434d906a00e06474c587" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40505 } } peers { permanent_uuid: "939f8ef107804ff18436c3374f446efc" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42503 } } peers { permanent_uuid: "5fa134a063644e579702cde50edb2da5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43805 } }
I20260812 07:12:31.964243 21546 leader_election.cc:288] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [CANDIDATE]: Term 1 election: Requested vote from peers 939f8ef107804ff18436c3374f446efc (127.31.156.65:42503), 5fa134a063644e579702cde50edb2da5 (127.31.156.66:43805)
I20260812 07:12:31.964944 21147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d176a97a76b4719bc7c364739c24ef6" candidate_uuid: "d0899e5ef5d8434d906a00e06474c587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939f8ef107804ff18436c3374f446efc"
I20260812 07:12:31.965261 21303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d176a97a76b4719bc7c364739c24ef6" candidate_uuid: "d0899e5ef5d8434d906a00e06474c587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa134a063644e579702cde50edb2da5"
I20260812 07:12:31.965296 21147 raft_consensus.cc:3037] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:31.965523 21303 raft_consensus.cc:3037] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:31.967394 21303 raft_consensus.cc:2454] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0899e5ef5d8434d906a00e06474c587 in term 1.
I20260812 07:12:31.967414 21147 raft_consensus.cc:2454] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0899e5ef5d8434d906a00e06474c587 in term 1.
I20260812 07:12:31.968236 21384 leader_election.cc:302] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5fa134a063644e579702cde50edb2da5, 939f8ef107804ff18436c3374f446efc, d0899e5ef5d8434d906a00e06474c587; no voters:
I20260812 07:12:31.968596 21546 raft_consensus.cc:2781] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:31.969133 21546 raft_consensus.cc:686] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 1 LEADER]: Becoming Leader. State: Replica: d0899e5ef5d8434d906a00e06474c587, State: Running, Role: LEADER
I20260812 07:12:31.969707 21546 consensus_queue.cc:237] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0899e5ef5d8434d906a00e06474c587" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40505 } } peers { permanent_uuid: "939f8ef107804ff18436c3374f446efc" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42503 } } peers { permanent_uuid: "5fa134a063644e579702cde50edb2da5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43805 } }
I20260812 07:12:31.976014 20984 catalog_manager.cc:5495] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0899e5ef5d8434d906a00e06474c587 (127.31.156.67). New cstate: current_term: 1 leader_uuid: "d0899e5ef5d8434d906a00e06474c587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0899e5ef5d8434d906a00e06474c587" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939f8ef107804ff18436c3374f446efc" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fa134a063644e579702cde50edb2da5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260812 07:12:32.036805 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:32.029885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43808 (local address 127.31.156.67:40505)
0812 07:12:32.030125 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:32.030131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:32.030158 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:12:32.030394 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:32.030398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:32.030501 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:32.030821 (+ 320us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:12:32.031252 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.032155 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.033910 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.034156 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.034319 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:32.035010 (+ 691us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:12:32.035811 (+ 801us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:12:32.036049 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:32.036438 (+ 389us) server_negotiation.cc:299] Negotiation successful
0812 07:12:32.036526 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 07:12:32.047889 21147 raft_consensus.cc:1261] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc [term 1 FOLLOWER]: Refusing update from remote peer d0899e5ef5d8434d906a00e06474c587: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:32.048120 21303 raft_consensus.cc:1261] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5 [term 1 FOLLOWER]: Refusing update from remote peer d0899e5ef5d8434d906a00e06474c587: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:32.049366 21546 consensus_queue.cc:1035] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939f8ef107804ff18436c3374f446efc" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 42503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:32.049783 21551 consensus_queue.cc:1035] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa134a063644e579702cde50edb2da5" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:32.066749 21556 mvcc.cc:205] Tried to move back new op lower bound from 7317580808373645312 to 7317580808073961472. Current Snapshot: MvccSnapshot[applied={T|T < 7317580808373645312}]
I20260812 07:12:32.066808 21557 mvcc.cc:205] Tried to move back new op lower bound from 7317580808373645312 to 7317580808073961472. Current Snapshot: MvccSnapshot[applied={T|T < 7317580808373645312}]
I20260812 07:12:32.085650 20984 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:32.085944 20984 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:32.090423 20984 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64e1c8ad16fd45728c460f07a73b376b: Sending DeleteTablet for 3 replicas of tablet 2d176a97a76b4719bc7c364739c24ef6
I20260812 07:12:32.091763 21279 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d176a97a76b4719bc7c364739c24ef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:32 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:37134
I20260812 07:12:32.091763 21127 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d176a97a76b4719bc7c364739c24ef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:32 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:51424
I20260812 07:12:32.091929 21443 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d176a97a76b4719bc7c364739c24ef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:32 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:43794
I20260812 07:12:32.093142 21568 tablet_replica.cc:330] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: stopping tablet replica
I20260812 07:12:32.093151 21567 tablet_replica.cc:330] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc: stopping tablet replica
I20260812 07:12:32.093830 21567 raft_consensus.cc:2229] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:32.094475 21567 raft_consensus.cc:2258] T 2d176a97a76b4719bc7c364739c24ef6 P 939f8ef107804ff18436c3374f446efc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:32.094575 21566 tablet_replica.cc:330] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: stopping tablet replica
I20260812 07:12:32.094609 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 21054
I20260812 07:12:32.105310 21566 raft_consensus.cc:2229] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:32.106076 21566 raft_consensus.cc:2258] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:32.093858 21568 raft_consensus.cc:2229] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:12:32.108444 21568 raft_consensus.cc:2258] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:32.111059 21566 ts_tablet_manager.cc:1905] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:32.114995 21566 ts_tablet_manager.cc:1918] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:32.115223 21566 log.cc:1196] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/2d176a97a76b4719bc7c364739c24ef6
I20260812 07:12:32.115835 21566 ts_tablet_manager.cc:1939] T 2d176a97a76b4719bc7c364739c24ef6 P 5fa134a063644e579702cde50edb2da5: Deleting consensus metadata
I20260812 07:12:32.117687 21568 ts_tablet_manager.cc:1905] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:32.118512 20969 catalog_manager.cc:4841] TS 5fa134a063644e579702cde50edb2da5 (127.31.156.66:43805): tablet 2d176a97a76b4719bc7c364739c24ef6 (table test-table [id=a4a233cc0e34420c92e9ae59278246a8]) successfully deleted
I20260812 07:12:32.121654 21568 ts_tablet_manager.cc:1918] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:32.121904 21568 log.cc:1196] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/2d176a97a76b4719bc7c364739c24ef6
W20260812 07:12:32.121876 20968 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:42503 (error 108)
I20260812 07:12:32.122507 21568 ts_tablet_manager.cc:1939] T 2d176a97a76b4719bc7c364739c24ef6 P d0899e5ef5d8434d906a00e06474c587: Deleting consensus metadata
I20260812 07:12:32.123051 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 21204
I20260812 07:12:32.138813 20971 catalog_manager.cc:4841] TS d0899e5ef5d8434d906a00e06474c587 (127.31.156.67:40505): tablet 2d176a97a76b4719bc7c364739c24ef6 (table test-table [id=a4a233cc0e34420c92e9ae59278246a8]) successfully deleted
I20260812 07:12:32.142230 21571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:32.141271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:42503 (local address 127.0.0.1:51438)
0812 07:12:32.141776 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:32.141961 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:42503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":158,"threads_started":1}
W20260812 07:12:32.142952 20968 catalog_manager.cc:4584] TS 939f8ef107804ff18436c3374f446efc (127.31.156.65:42503): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d176a97a76b4719bc7c364739c24ef6: Network error: Client connection negotiation failed: client connection to 127.31.156.65:42503: connect: Connection refused (error 111)
I20260812 07:12:32.151266 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 21365
I20260812 07:12:32.169746 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 20950
2026-08-12T07:12:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (3161 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T07:12:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:32Z Disabled control of system clock
I20260812 07:12:32.231822 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:39845
--webserver_interface=127.31.156.126
--webserver_port=0
--builtin_ntp_servers=127.31.156.84:43119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:39845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:12:32.410249 21578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:32.410607 21578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:32.410647 21578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:32.415045 21578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:32.415141 21578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:32.415160 21578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:32.415179 21578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:32.415194 21578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:32.420434 21578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:43119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:39845
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:39845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:32.421649 21578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:32.423183 21578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:32.433318 21584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:32.433318 21583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:32.433318 21586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:32.433761 21578 server_base.cc:1034] running on GCE node
I20260812 07:12:32.434468 21578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:32.436265 21578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:32.437490 21578 hybrid_clock.cc:648] HybridClock initialized: now 1786518752437450 us; error 60 us; skew 500 ppm
I20260812 07:12:32.440236 21578 webserver.cc:460] Webserver started at http://127.31.156.126:36699/ using document root <none> and password file <none>
I20260812 07:12:32.440968 21578 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:32.441049 21578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:32.441358 21578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:32.444018 21578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "72843ca97020496c832645a8c6844680"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-csg5"
server_key: "3d38bff0dd393bcd6fd07986c4afa494"
server_key_iv: "b8a685e2522fb98734650565432b73ce"
server_key_version: "encryptionkey@0"
I20260812 07:12:32.444756 21578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "72843ca97020496c832645a8c6844680"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-csg5"
server_key: "3d38bff0dd393bcd6fd07986c4afa494"
server_key_iv: "b8a685e2522fb98734650565432b73ce"
server_key_version: "encryptionkey@0"
I20260812 07:12:32.450091 21578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:12:32.453568 21592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.455451 21578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:32.455636 21578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "72843ca97020496c832645a8c6844680"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-csg5"
server_key: "3d38bff0dd393bcd6fd07986c4afa494"
server_key_iv: "b8a685e2522fb98734650565432b73ce"
server_key_version: "encryptionkey@0"
I20260812 07:12:32.455791 21578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:32.473829 21578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:32.474735 21578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:32.474959 21578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:32.483176 21578 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:39845
I20260812 07:12:32.483193 21651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:39845 every 8 connection(s)
I20260812 07:12:32.484390 21578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:32.486493 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 21578
I20260812 07:12:32.486672 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:32.487018 32369 external_mini_cluster.cc:1428] Setting key 171295daf71311e745fa53acee858ebe
I20260812 07:12:32.488624 21652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:32.495551 21652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680: Bootstrap starting.
I20260812 07:12:32.497587 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:32.488643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.31.156.126:39845)
0812 07:12:32.489411 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:32.489429 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:32.489470 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:12:32.490152 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:32.490169 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:32.490577 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:32.491003 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:32.491508 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.492544 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.493501 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.494004 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.494202 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:32.495475 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:32.495481 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:32.495497 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:32.496067 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:32.496139 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:32.496163 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:32.496294 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:32.496443 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 07:12:32.496760 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":184,"threads_started":1}
I20260812 07:12:32.498646 21652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:32.499603 21652 log.cc:824] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:32.502038 21652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680: No bootstrap required, opened a new log
I20260812 07:12:32.506284 21652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72843ca97020496c832645a8c6844680" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39845 } }
I20260812 07:12:32.506688 21652 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:32.506795 21652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72843ca97020496c832645a8c6844680, State: Initialized, Role: FOLLOWER
I20260812 07:12:32.507331 21652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72843ca97020496c832645a8c6844680" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39845 } }
I20260812 07:12:32.507495 21652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:32.507591 21652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:32.507700 21652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:32.508827 21652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72843ca97020496c832645a8c6844680" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39845 } }
I20260812 07:12:32.509275 21652 leader_election.cc:302] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72843ca97020496c832645a8c6844680; no voters:
I20260812 07:12:32.509673 21652 leader_election.cc:288] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:32.509945 21657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:32.510246 21657 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [term 1 LEADER]: Becoming Leader. State: Replica: 72843ca97020496c832645a8c6844680, State: Running, Role: LEADER
I20260812 07:12:32.510634 21657 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72843ca97020496c832645a8c6844680" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39845 } }
I20260812 07:12:32.511315 21652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:32.512746 21658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72843ca97020496c832645a8c6844680" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72843ca97020496c832645a8c6844680" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39845 } } }
I20260812 07:12:32.512912 21658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:32.513180 21659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72843ca97020496c832645a8c6844680. Latest consensus state: current_term: 1 leader_uuid: "72843ca97020496c832645a8c6844680" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72843ca97020496c832645a8c6844680" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39845 } } }
I20260812 07:12:32.513381 21659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:32.513432 21668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:32.517374 21668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:32.524842 21668 catalog_manager.cc:1347] Generated new cluster ID: d4446e32f2414dd19a831851e6f8a5c5
I20260812 07:12:32.524942 21668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:32.535142 21668 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:32.536697 21668 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:32.562574 21668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680: Generated new TSK 0
I20260812 07:12:32.563525 21668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:32.574954 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39845
--builtin_ntp_servers=127.31.156.84:43119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:12:32.702257 21677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:32.702658 21677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:32.702759 21677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:32.707211 21677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:32.707331 21677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:32.707429 21677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:12:32.712937 21677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:43119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:32.714313 21677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:32.715943 21677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:32.727775 21682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:32.727841 21683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:32.727845 21685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:32.728078 21677 server_base.cc:1034] running on GCE node
I20260812 07:12:32.728714 21677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:32.729553 21677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:32.730770 21677 hybrid_clock.cc:648] HybridClock initialized: now 1786518752730755 us; error 33 us; skew 500 ppm
I20260812 07:12:32.733013 21677 webserver.cc:460] Webserver started at http://127.31.156.65:36243/ using document root <none> and password file <none>
I20260812 07:12:32.733712 21677 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:32.733834 21677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:32.734093 21677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:32.736503 21677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "fe2cfa2c66b44cc28633182199fcce03"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-csg5"
server_key: "24153f40c2c9d200c87d0cf9300bcc83"
server_key_iv: "bc9bcb98efd0225ff043cefd163475f1"
server_key_version: "encryptionkey@0"
I20260812 07:12:32.737224 21677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "fe2cfa2c66b44cc28633182199fcce03"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-csg5"
server_key: "24153f40c2c9d200c87d0cf9300bcc83"
server_key_iv: "bc9bcb98efd0225ff043cefd163475f1"
server_key_version: "encryptionkey@0"
I20260812 07:12:32.741693 21677 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:12:32.744716 21692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.746191 21677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:12:32.746364 21677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "fe2cfa2c66b44cc28633182199fcce03"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-csg5"
server_key: "24153f40c2c9d200c87d0cf9300bcc83"
server_key_iv: "bc9bcb98efd0225ff043cefd163475f1"
server_key_version: "encryptionkey@0"
I20260812 07:12:32.746510 21677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:32.772040 21677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:32.772846 21677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:32.773072 21677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:32.773738 21677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:32.774796 21677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:32.774914 21677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.774991 21677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:32.775035 21677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.785245 21677 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:41445
I20260812 07:12:32.785301 21827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:41445 every 8 connection(s)
I20260812 07:12:32.786320 21677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:32.794687 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 21677
I20260812 07:12:32.794847 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:12:32.795156 32369 external_mini_cluster.cc:1428] Setting key 0e3f156ae8e3f82ae25726d31a21e6a9
I20260812 07:12:32.797533 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39845
--builtin_ntp_servers=127.31.156.84:43119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:12:32.798914 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:32.788883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:53471 (local address 127.31.156.126:39845)
0812 07:12:32.789049 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:32.789053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:32.790426 (+ 1373us) server_negotiation.cc:407] Connection header received
0812 07:12:32.792315 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:32.792318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:32.792381 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:32.792473 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:32.793686 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.794439 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.796356 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.796621 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.796810 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:32.798282 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:32.798286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:32.798288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:32.798303 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:32.798326 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:32.798335 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:32.798433 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:32.798622 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 07:12:32.798744 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:12:32.800019 21830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:32.789362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:39845 (local address 127.31.156.65:53471)
0812 07:12:32.790147 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:32.790286 (+ 139us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:32.791881 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:32.792747 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:32.792761 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:32.792858 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:32.793493 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:32.793509 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.794625 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.794628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:32.796113 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:32.796127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.796873 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.796878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:32.797087 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:32.797907 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:12:32.797909 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:32.797966 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:12:32.798055 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:32.798437 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:32.798452 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:32.798459 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:12:32.798625 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 07:12:32.799067 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":149,"threads_started":1}
I20260812 07:12:32.801589 21828 heartbeater.cc:347] Connected to a master server at 127.31.156.126:39845
I20260812 07:12:32.801899 21828 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:32.802706 21828 heartbeater.cc:511] Master 127.31.156.126:39845 requested a full tablet report, sending...
I20260812 07:12:32.804809 21612 ts_manager.cc:194] Registered new tserver with Master: fe2cfa2c66b44cc28633182199fcce03 (127.31.156.65:41445)
I20260812 07:12:32.806103 21612 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.65:53471
W20260812 07:12:32.925997 21831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:32.926304 21831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:32.926347 21831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:32.930850 21831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:32.930935 21831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:32.930996 21831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:12:32.936223 21831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:43119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:32.937516 21831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:32.939083 21831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:32.950994 21839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:32.951071 21838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:32.951084 21831 server_base.cc:1034] running on GCE node
W20260812 07:12:32.950994 21841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:32.951855 21831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:32.952716 21831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:32.953969 21831 hybrid_clock.cc:648] HybridClock initialized: now 1786518752953902 us; error 82 us; skew 500 ppm
I20260812 07:12:32.956383 21831 webserver.cc:460] Webserver started at http://127.31.156.66:37395/ using document root <none> and password file <none>
I20260812 07:12:32.957000 21831 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:32.957067 21831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:32.957269 21831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:32.959767 21831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "73165a79d52e4b0a993af0d94829d7d2"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-csg5"
server_key: "8bfe93006b4a8d959e1d89dc4c3634ef"
server_key_iv: "fa2fbbee709454e232d833434465a607"
server_key_version: "encryptionkey@0"
I20260812 07:12:32.960533 21831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "73165a79d52e4b0a993af0d94829d7d2"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-csg5"
server_key: "8bfe93006b4a8d959e1d89dc4c3634ef"
server_key_iv: "fa2fbbee709454e232d833434465a607"
server_key_version: "encryptionkey@0"
I20260812 07:12:32.964967 21831 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:12:32.968106 21847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.969977 21831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:12:32.970153 21831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "73165a79d52e4b0a993af0d94829d7d2"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-csg5"
server_key: "8bfe93006b4a8d959e1d89dc4c3634ef"
server_key_iv: "fa2fbbee709454e232d833434465a607"
server_key_version: "encryptionkey@0"
I20260812 07:12:32.970278 21831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:32.990711 21831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:32.991508 21831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:32.991684 21831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:32.992290 21831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:32.993417 21831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:32.993469 21831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.993513 21831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:32.993579 21831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:33.004084 21831 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:40853
I20260812 07:12:33.004755 21987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:40853 every 8 connection(s)
I20260812 07:12:33.005681 21831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:12:33.007323 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 21831
I20260812 07:12:33.007459 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:12:33.007885 32369 external_mini_cluster.cc:1428] Setting key a1d4b92a4160a7bfb437a3f6661c1ec5
I20260812 07:12:33.010325 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39845
--builtin_ntp_servers=127.31.156.84:43119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:12:33.018013 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.008032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:34709 (local address 127.31.156.126:39845)
0812 07:12:33.008220 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:33.008234 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:33.009585 (+ 1351us) server_negotiation.cc:407] Connection header received
0812 07:12:33.011267 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:33.011272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:33.011332 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:33.011421 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:33.012662 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.013412 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.015427 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.015696 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.015809 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.017444 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:33.017447 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:33.017450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:33.017466 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:33.017491 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:33.017499 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:33.017572 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.017799 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.017872 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:12:33.019660 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.008595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:39845 (local address 127.31.156.66:34709)
0812 07:12:33.009411 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:33.009474 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:33.011020 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:33.011629 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:33.011637 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:33.011803 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:33.012486 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.012511 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.013591 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.013597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.015222 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.015240 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.015918 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.015927 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.016204 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.017028 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:12:33.017032 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:33.017101 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:12:33.017215 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:33.017602 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:33.017616 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:33.017624 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:12:33.017791 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 07:12:33.018319 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":159,"threads_started":1}
I20260812 07:12:33.021328 21988 heartbeater.cc:347] Connected to a master server at 127.31.156.126:39845
I20260812 07:12:33.021615 21988 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:33.022240 21988 heartbeater.cc:511] Master 127.31.156.126:39845 requested a full tablet report, sending...
I20260812 07:12:33.023931 21612 ts_manager.cc:194] Registered new tserver with Master: 73165a79d52e4b0a993af0d94829d7d2 (127.31.156.66:40853)
I20260812 07:12:33.024585 21612 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.66:34709
W20260812 07:12:33.140147 21992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:33.140542 21992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:33.140619 21992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:33.144927 21992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:33.145049 21992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:33.145169 21992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:12:33.150467 21992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:43119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:33.151830 21992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:33.153437 21992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:33.164774 22003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:33.164989 21992 server_base.cc:1034] running on GCE node
W20260812 07:12:33.164844 22001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:33.165236 21998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:33.165802 21992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:33.166558 21992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:33.167853 21992 hybrid_clock.cc:648] HybridClock initialized: now 1786518753167833 us; error 39 us; skew 500 ppm
I20260812 07:12:33.170113 21992 webserver.cc:460] Webserver started at http://127.31.156.67:41061/ using document root <none> and password file <none>
I20260812 07:12:33.170881 21992 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:33.170987 21992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:33.171247 21992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:33.173525 21992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "b9108569db49475499fe5c1b4bf8fd71"
format_stamp: "Formatted at 2026-08-12 07:12:33 on dist-test-slave-csg5"
server_key: "9bf67b2a89f9a43431af8954d1855ef4"
server_key_iv: "11a78ddd9916700d72acb09a6668c0e5"
server_key_version: "encryptionkey@0"
I20260812 07:12:33.174242 21992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "b9108569db49475499fe5c1b4bf8fd71"
format_stamp: "Formatted at 2026-08-12 07:12:33 on dist-test-slave-csg5"
server_key: "9bf67b2a89f9a43431af8954d1855ef4"
server_key_iv: "11a78ddd9916700d72acb09a6668c0e5"
server_key_version: "encryptionkey@0"
I20260812 07:12:33.178520 21992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260812 07:12:33.181429 22009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:33.182935 21992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:33.183091 21992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "b9108569db49475499fe5c1b4bf8fd71"
format_stamp: "Formatted at 2026-08-12 07:12:33 on dist-test-slave-csg5"
server_key: "9bf67b2a89f9a43431af8954d1855ef4"
server_key_iv: "11a78ddd9916700d72acb09a6668c0e5"
server_key_version: "encryptionkey@0"
I20260812 07:12:33.183229 21992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:33.215878 21992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:33.216889 21992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:33.217185 21992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:33.218050 21992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:33.219275 21992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:33.219359 21992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:33.219447 21992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:33.219497 21992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:33.231024 21992 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:41629
I20260812 07:12:33.231148 22145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:41629 every 8 connection(s)
I20260812 07:12:33.232178 21992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:33.241387 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 21992
I20260812 07:12:33.241529 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:12:33.241899 32369 external_mini_cluster.cc:1428] Setting key b1dc5100a3d38e1e1b85a37efbaf74de
I20260812 07:12:33.244396 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.234748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:50765 (local address 127.31.156.126:39845)
0812 07:12:33.234935 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:33.234939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:33.236425 (+ 1486us) server_negotiation.cc:407] Connection header received
0812 07:12:33.238108 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:33.238112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:33.238172 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:33.238281 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:33.239544 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.240190 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.241874 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.242178 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.242314 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.243770 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:33.243775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:33.243779 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:33.243794 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:33.243830 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:33.243843 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:33.243929 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.244149 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.244245 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:12:33.245489 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.235425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:39845 (local address 127.31.156.67:50765)
0812 07:12:33.236206 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:33.236261 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:33.237844 (+ 1583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:33.238491 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:33.238500 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:33.238593 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:33.239343 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.239361 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.240349 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.240352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.241716 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.241731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.242361 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.242364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.242532 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.243391 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:12:33.243393 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:33.243450 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:12:33.243529 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:33.243950 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:33.243961 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:33.243968 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:12:33.244103 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 07:12:33.244521 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":130,"threads_started":1}
I20260812 07:12:33.246941 22146 heartbeater.cc:347] Connected to a master server at 127.31.156.126:39845
I20260812 07:12:33.247223 22146 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:33.247838 22146 heartbeater.cc:511] Master 127.31.156.126:39845 requested a full tablet report, sending...
I20260812 07:12:33.249167 21612 ts_manager.cc:194] Registered new tserver with Master: b9108569db49475499fe5c1b4bf8fd71 (127.31.156.67:41629)
I20260812 07:12:33.249949 21612 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.67:50765
I20260812 07:12:33.257462 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:12:33.264681 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.260014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.31.156.126:39845)
0812 07:12:33.260181 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:33.260186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:33.260513 (+ 327us) server_negotiation.cc:407] Connection header received
0812 07:12:33.260578 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:33.260583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:33.260643 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:33.260720 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:33.261049 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.261702 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.262728 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.263001 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.263109 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.264048 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:33.264052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:33.264054 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:33.264067 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:33.264088 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:33.264096 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:33.264328 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.264422 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.264458 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:12:33.268653 21612 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:12:33.272285 21612 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:12:33.291188 22162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.283864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52498 (local address 127.31.156.65:41445)
0812 07:12:33.284395 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:33.284402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:33.284827 (+ 425us) server_negotiation.cc:407] Connection header received
0812 07:12:33.285212 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:33.285403 (+ 191us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:33.285687 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:33.285850 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:33.286379 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.287670 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.289390 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.290192 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.290310 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.290546 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.290793 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.290955 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":136,"threads_started":1}
I20260812 07:12:33.291428 22163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.284575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:40853 (local address 127.0.0.1:35146)
0812 07:12:33.285042 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:33.285059 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:33.285163 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:33.285970 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:33.285973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:33.285986 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:33.286199 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.286208 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.287766 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.287770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.289602 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.289610 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.290702 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.290709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.290890 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.290916 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 07:12:33.290986 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 07:12:33.291202 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":193,"thread_start_us":143,"threads_started":1}
I20260812 07:12:33.291437 22167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.284561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.31.156.67:41629)
0812 07:12:33.285596 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:33.285609 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:33.285633 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:12:33.285709 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:33.285714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:33.285898 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:33.286052 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:33.286382 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.287623 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.289759 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.290546 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.290695 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.290899 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.291045 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.291149 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":928,"thread_start_us":181,"threads_started":1}
I20260812 07:12:33.291640 22161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.283717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:41445 (local address 127.0.0.1:52498)
0812 07:12:33.284647 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:33.284709 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:33.284962 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:33.285884 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:33.285893 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:33.285926 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:33.286235 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.286248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.287792 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.287795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.289252 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.289262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.290345 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.290349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.290480 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.290542 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 07:12:33.291395 (+ 853us) client_negotiation.cc:240] Negotiation successful
0812 07:12:33.291486 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":326,"thread_start_us":123,"threads_started":1}
I20260812 07:12:33.291934 22164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.283942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:41629 (local address 127.0.0.1:59610)
0812 07:12:33.284853 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:33.284869 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:33.284970 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:33.286101 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:33.286104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:33.286117 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:33.286251 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.286261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.287983 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.287987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.289597 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.289610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.290712 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.290716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.290895 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.290927 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:12:33.291001 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 07:12:33.291192 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":255,"thread_start_us":135,"threads_started":1}
I20260812 07:12:33.293273 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.285058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.31.156.66:40853)
0812 07:12:33.285457 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:33.285472 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:33.285499 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:12:33.285577 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:33.285583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:33.285814 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:33.286086 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:33.286337 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.287601 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.289796 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.290489 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.292652 (+ 2163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.292901 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.292976 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.293094 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":166,"threads_started":1}
I20260812 07:12:33.295408 21750 tablet_service.cc:1467] Processing CreateTablet for tablet af38a7efdf7545c1bdaee0241ae01a9c (DEFAULT_TABLE table=test-table [id=8c14645a98f748a8933b4a5ace14ecab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:33.295751 22065 tablet_service.cc:1467] Processing CreateTablet for tablet af38a7efdf7545c1bdaee0241ae01a9c (DEFAULT_TABLE table=test-table [id=8c14645a98f748a8933b4a5ace14ecab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:33.296447 21905 tablet_service.cc:1467] Processing CreateTablet for tablet af38a7efdf7545c1bdaee0241ae01a9c (DEFAULT_TABLE table=test-table [id=8c14645a98f748a8933b4a5ace14ecab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:33.297045 22065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af38a7efdf7545c1bdaee0241ae01a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:33.297075 21750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af38a7efdf7545c1bdaee0241ae01a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:33.297546 21905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af38a7efdf7545c1bdaee0241ae01a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:33.305037 22170 tablet_bootstrap.cc:492] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03: Bootstrap starting.
I20260812 07:12:33.306152 22172 tablet_bootstrap.cc:492] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: Bootstrap starting.
I20260812 07:12:33.307117 22171 tablet_bootstrap.cc:492] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: Bootstrap starting.
I20260812 07:12:33.308774 22170 tablet_bootstrap.cc:654] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:33.308796 22172 tablet_bootstrap.cc:654] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:33.309489 22171 tablet_bootstrap.cc:654] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:33.309685 22170 log.cc:824] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:33.309764 22172 log.cc:824] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:33.310461 22171 log.cc:824] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:33.312006 22170 tablet_bootstrap.cc:492] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03: No bootstrap required, opened a new log
I20260812 07:12:33.312297 22170 ts_tablet_manager.cc:1397] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 07:12:33.312551 22171 tablet_bootstrap.cc:492] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: No bootstrap required, opened a new log
I20260812 07:12:33.312893 22171 ts_tablet_manager.cc:1397] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 07:12:33.316977 22170 raft_consensus.cc:348] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2cfa2c66b44cc28633182199fcce03" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41445 } } peers { permanent_uuid: "73165a79d52e4b0a993af0d94829d7d2" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 40853 } } peers { permanent_uuid: "b9108569db49475499fe5c1b4bf8fd71" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41629 } }
I20260812 07:12:33.316946 22171 raft_consensus.cc:348] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2cfa2c66b44cc28633182199fcce03" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41445 } } peers { permanent_uuid: "73165a79d52e4b0a993af0d94829d7d2" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 40853 } } peers { permanent_uuid: "b9108569db49475499fe5c1b4bf8fd71" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41629 } }
I20260812 07:12:33.317258 22170 raft_consensus.cc:374] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:33.317250 22171 raft_consensus.cc:374] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:33.317329 22170 raft_consensus.cc:729] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2cfa2c66b44cc28633182199fcce03, State: Initialized, Role: FOLLOWER
I20260812 07:12:33.317341 22171 raft_consensus.cc:729] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9108569db49475499fe5c1b4bf8fd71, State: Initialized, Role: FOLLOWER
I20260812 07:12:33.317852 22171 consensus_queue.cc:260] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2cfa2c66b44cc28633182199fcce03" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41445 } } peers { permanent_uuid: "73165a79d52e4b0a993af0d94829d7d2" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 40853 } } peers { permanent_uuid: "b9108569db49475499fe5c1b4bf8fd71" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41629 } }
I20260812 07:12:33.317855 22170 consensus_queue.cc:260] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2cfa2c66b44cc28633182199fcce03" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41445 } } peers { permanent_uuid: "73165a79d52e4b0a993af0d94829d7d2" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 40853 } } peers { permanent_uuid: "b9108569db49475499fe5c1b4bf8fd71" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41629 } }
I20260812 07:12:33.318691 21828 heartbeater.cc:503] Master 127.31.156.126:39845 was elected leader, sending a full tablet report...
I20260812 07:12:33.319096 22171 ts_tablet_manager.cc:1428] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: Time spent starting tablet: real 0.006s user 0.002s sys 0.006s
I20260812 07:12:33.319105 22170 ts_tablet_manager.cc:1428] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260812 07:12:33.319942 22146 heartbeater.cc:503] Master 127.31.156.126:39845 was elected leader, sending a full tablet report...
I20260812 07:12:33.321071 22172 tablet_bootstrap.cc:492] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: No bootstrap required, opened a new log
I20260812 07:12:33.321338 22172 ts_tablet_manager.cc:1397] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260812 07:12:33.326359 22172 raft_consensus.cc:348] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2cfa2c66b44cc28633182199fcce03" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41445 } } peers { permanent_uuid: "73165a79d52e4b0a993af0d94829d7d2" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 40853 } } peers { permanent_uuid: "b9108569db49475499fe5c1b4bf8fd71" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41629 } }
I20260812 07:12:33.326637 22172 raft_consensus.cc:374] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:33.326741 22172 raft_consensus.cc:729] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73165a79d52e4b0a993af0d94829d7d2, State: Initialized, Role: FOLLOWER
I20260812 07:12:33.327205 22172 consensus_queue.cc:260] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2cfa2c66b44cc28633182199fcce03" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41445 } } peers { permanent_uuid: "73165a79d52e4b0a993af0d94829d7d2" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 40853 } } peers { permanent_uuid: "b9108569db49475499fe5c1b4bf8fd71" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41629 } }
I20260812 07:12:33.327951 21988 heartbeater.cc:503] Master 127.31.156.126:39845 was elected leader, sending a full tablet report...
I20260812 07:12:33.328378 22172 ts_tablet_manager.cc:1428] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260812 07:12:33.483557 22148 tablet.cc:2369] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:12:33.505649 22177 raft_consensus.cc:482] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:33.505963 22177 raft_consensus.cc:504] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2cfa2c66b44cc28633182199fcce03" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41445 } } peers { permanent_uuid: "73165a79d52e4b0a993af0d94829d7d2" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 40853 } } peers { permanent_uuid: "b9108569db49475499fe5c1b4bf8fd71" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41629 } }
W20260812 07:12:33.507089 21989 tablet.cc:2369] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:12:33.507467 22177 leader_election.cc:288] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe2cfa2c66b44cc28633182199fcce03 (127.31.156.65:41445), 73165a79d52e4b0a993af0d94829d7d2 (127.31.156.66:40853)
I20260812 07:12:33.512966 22162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.507902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:50873 (local address 127.31.156.65:41445)
0812 07:12:33.508077 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:33.508082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:33.508102 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:12:33.508395 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:33.508398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:33.508462 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:33.508556 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:33.508983 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.509835 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.511370 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.512083 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.512289 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.512334 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.512685 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.512783 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:12:33.512952 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.507766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:41445 (local address 127.31.156.67:50873)
0812 07:12:33.507934 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:33.507954 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:33.508100 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:33.508597 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:33.508599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:33.508620 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:33.508830 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.508839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.510061 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.510065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.511207 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.511219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.512393 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.512396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.512523 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.512543 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 07:12:33.512642 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 07:12:33.512736 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 07:12:33.513139 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.508261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:34869 (local address 127.31.156.66:40853)
0812 07:12:33.508401 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:33.508405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:33.508697 (+ 292us) server_negotiation.cc:407] Connection header received
0812 07:12:33.508801 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:33.508805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:33.508864 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:33.508954 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:33.509336 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.510336 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.511954 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.512512 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.512666 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.512727 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.512921 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.513017 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:12:33.513231 22179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.508115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:40853 (local address 127.31.156.67:34869)
0812 07:12:33.508597 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:33.508610 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:33.508714 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:33.508974 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:33.508977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:33.508994 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:33.509111 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.509117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.510483 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.510485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.511783 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:33.511794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.512645 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.512649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:33.512736 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.512757 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 07:12:33.512875 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 07:12:33.512934 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":173,"threads_started":1}
I20260812 07:12:33.514029 21933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af38a7efdf7545c1bdaee0241ae01a9c" candidate_uuid: "b9108569db49475499fe5c1b4bf8fd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73165a79d52e4b0a993af0d94829d7d2" is_pre_election: true
I20260812 07:12:33.514038 21773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af38a7efdf7545c1bdaee0241ae01a9c" candidate_uuid: "b9108569db49475499fe5c1b4bf8fd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2cfa2c66b44cc28633182199fcce03" is_pre_election: true
I20260812 07:12:33.514417 21773 raft_consensus.cc:2454] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9108569db49475499fe5c1b4bf8fd71 in term 0.
I20260812 07:12:33.514451 21933 raft_consensus.cc:2454] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9108569db49475499fe5c1b4bf8fd71 in term 0.
I20260812 07:12:33.515064 22012 leader_election.cc:302] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73165a79d52e4b0a993af0d94829d7d2, b9108569db49475499fe5c1b4bf8fd71; no voters:
I20260812 07:12:33.515422 22177 raft_consensus.cc:2781] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:12:33.515522 22177 raft_consensus.cc:482] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:12:33.515570 22177 raft_consensus.cc:3037] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:33.516697 22177 raft_consensus.cc:504] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2cfa2c66b44cc28633182199fcce03" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41445 } } peers { permanent_uuid: "73165a79d52e4b0a993af0d94829d7d2" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 40853 } } peers { permanent_uuid: "b9108569db49475499fe5c1b4bf8fd71" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41629 } }
I20260812 07:12:33.517097 22177 leader_election.cc:288] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [CANDIDATE]: Term 1 election: Requested vote from peers fe2cfa2c66b44cc28633182199fcce03 (127.31.156.65:41445), 73165a79d52e4b0a993af0d94829d7d2 (127.31.156.66:40853)
I20260812 07:12:33.517670 21773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af38a7efdf7545c1bdaee0241ae01a9c" candidate_uuid: "b9108569db49475499fe5c1b4bf8fd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2cfa2c66b44cc28633182199fcce03"
I20260812 07:12:33.517678 21933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af38a7efdf7545c1bdaee0241ae01a9c" candidate_uuid: "b9108569db49475499fe5c1b4bf8fd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73165a79d52e4b0a993af0d94829d7d2"
I20260812 07:12:33.517845 21773 raft_consensus.cc:3037] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:33.517877 21933 raft_consensus.cc:3037] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:33.519222 21773 raft_consensus.cc:2454] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9108569db49475499fe5c1b4bf8fd71 in term 1.
I20260812 07:12:33.519310 21933 raft_consensus.cc:2454] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9108569db49475499fe5c1b4bf8fd71 in term 1.
I20260812 07:12:33.519680 22012 leader_election.cc:302] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9108569db49475499fe5c1b4bf8fd71, fe2cfa2c66b44cc28633182199fcce03; no voters:
I20260812 07:12:33.519893 22177 raft_consensus.cc:2781] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:33.520140 22177 raft_consensus.cc:686] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 1 LEADER]: Becoming Leader. State: Replica: b9108569db49475499fe5c1b4bf8fd71, State: Running, Role: LEADER
I20260812 07:12:33.520555 22177 consensus_queue.cc:237] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2cfa2c66b44cc28633182199fcce03" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41445 } } peers { permanent_uuid: "73165a79d52e4b0a993af0d94829d7d2" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 40853 } } peers { permanent_uuid: "b9108569db49475499fe5c1b4bf8fd71" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41629 } }
I20260812 07:12:33.524238 21608 catalog_manager.cc:5495] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9108569db49475499fe5c1b4bf8fd71 (127.31.156.67). New cstate: current_term: 1 leader_uuid: "b9108569db49475499fe5c1b4bf8fd71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2cfa2c66b44cc28633182199fcce03" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73165a79d52e4b0a993af0d94829d7d2" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9108569db49475499fe5c1b4bf8fd71" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 41629 } health_report { overall_health: HEALTHY } } }
W20260812 07:12:33.538089 21829 tablet.cc:2369] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:12:33.592739 22167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.588406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59624 (local address 127.31.156.67:41629)
0812 07:12:33.588551 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:33.588555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:33.588569 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:12:33.588648 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:33.588650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:33.588717 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:33.588813 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:12:33.589253 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.589779 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.590824 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.591084 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.591195 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.591631 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:12:33.592113 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:12:33.592328 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.592407 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.592451 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:12:33.599788 21773 raft_consensus.cc:1261] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03 [term 1 FOLLOWER]: Refusing update from remote peer b9108569db49475499fe5c1b4bf8fd71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:33.599788 21933 raft_consensus.cc:1261] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2 [term 1 FOLLOWER]: Refusing update from remote peer b9108569db49475499fe5c1b4bf8fd71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:33.600710 22177 consensus_queue.cc:1035] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73165a79d52e4b0a993af0d94829d7d2" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:33.601068 22180 consensus_queue.cc:1035] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe2cfa2c66b44cc28633182199fcce03" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:33.610437 22184 mvcc.cc:205] Tried to move back new op lower bound from 7317580814735654912 to 7317580814423453696. Current Snapshot: MvccSnapshot[applied={T|T < 7317580814735654912}]
I20260812 07:12:33.619395 21612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:33.619625 21612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:33.622824 21612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72843ca97020496c832645a8c6844680: Sending DeleteTablet for 3 replicas of tablet af38a7efdf7545c1bdaee0241ae01a9c
I20260812 07:12:33.623920 22065 tablet_service.cc:1514] Processing DeleteTablet for tablet af38a7efdf7545c1bdaee0241ae01a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:33 UTC) from {username='slave'} at 127.0.0.1:59610
I20260812 07:12:33.624101 21750 tablet_service.cc:1514] Processing DeleteTablet for tablet af38a7efdf7545c1bdaee0241ae01a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:33 UTC) from {username='slave'} at 127.0.0.1:52498
I20260812 07:12:33.624205 21905 tablet_service.cc:1514] Processing DeleteTablet for tablet af38a7efdf7545c1bdaee0241ae01a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:33 UTC) from {username='slave'} at 127.0.0.1:35146
I20260812 07:12:33.624581 22195 tablet_replica.cc:330] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: stopping tablet replica
I20260812 07:12:33.624869 22197 tablet_replica.cc:330] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: stopping tablet replica
I20260812 07:12:33.624997 22195 raft_consensus.cc:2229] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:12:33.625200 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 21677
I20260812 07:12:33.625209 22196 tablet_replica.cc:330] T af38a7efdf7545c1bdaee0241ae01a9c P fe2cfa2c66b44cc28633182199fcce03: stopping tablet replica
I20260812 07:12:33.631672 22195 raft_consensus.cc:2258] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:33.634193 22195 ts_tablet_manager.cc:1905] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:33.637106 22195 ts_tablet_manager.cc:1918] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:33.637296 22195 log.cc:1196] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/af38a7efdf7545c1bdaee0241ae01a9c
I20260812 07:12:33.637758 22195 ts_tablet_manager.cc:1939] T af38a7efdf7545c1bdaee0241ae01a9c P b9108569db49475499fe5c1b4bf8fd71: Deleting consensus metadata
I20260812 07:12:33.638909 22197 raft_consensus.cc:2229] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:33.639360 22197 raft_consensus.cc:2258] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:33.639456 21593 catalog_manager.cc:4841] TS b9108569db49475499fe5c1b4bf8fd71 (127.31.156.67:41629): tablet af38a7efdf7545c1bdaee0241ae01a9c (table test-table [id=8c14645a98f748a8933b4a5ace14ecab]) successfully deleted
I20260812 07:12:33.643885 22197 ts_tablet_manager.cc:1905] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 07:12:33.646081 21597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:41445 (error 108)
I20260812 07:12:33.647166 22197 ts_tablet_manager.cc:1918] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:33.647300 22197 log.cc:1196] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/af38a7efdf7545c1bdaee0241ae01a9c
I20260812 07:12:33.647502 22164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.647143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:41445 (local address 127.0.0.1:52514)
0812 07:12:33.647297 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:33.647386 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:41445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:12:33.647583 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 21831
I20260812 07:12:33.647645 22197 ts_tablet_manager.cc:1939] T af38a7efdf7545c1bdaee0241ae01a9c P 73165a79d52e4b0a993af0d94829d7d2: Deleting consensus metadata
W20260812 07:12:33.651209 21597 catalog_manager.cc:4584] TS fe2cfa2c66b44cc28633182199fcce03 (127.31.156.65:41445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af38a7efdf7545c1bdaee0241ae01a9c: Network error: Client connection negotiation failed: client connection to 127.31.156.65:41445: connect: Connection refused (error 111)
I20260812 07:12:33.657647 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 21992
I20260812 07:12:33.659448 22164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.657881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:40853 (local address 127.0.0.1:35160)
0812 07:12:33.659143 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:33.659217 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.66:40853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1161}
I20260812 07:12:33.668954 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 21578
2026-08-12T07:12:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1486 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:33 dist-test-slave-csg5 krb5kdc[22203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:33 dist-test-slave-csg5 krb5kdc[22203](info): set up 2 sockets
Aug 12 07:12:33 dist-test-slave-csg5 krb5kdc[22203](info): commencing operation
krb5kdc: starting...
W20260812 07:12:34.846134 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:34 dist-test-slave-csg5 krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:12:35.012293 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46517
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:46517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:35.139619 22218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:35.140017 22218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:35.140105 22218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:35.144599 22218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:35.144719 22218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:35.144752 22218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:35.144776 22218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:35.144804 22218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:35.151232 22218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:46517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:35.152616 22218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:35.153973 22218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:35.163995 22229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:35.163995 22226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:35.163995 22225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:35.164655 22218 server_base.cc:1034] running on GCE node
I20260812 07:12:35.165416 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:35.166792 22218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:35.168020 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786518755167992 us; error 55 us; skew 500 ppm
Aug 12 07:12:35 dist-test-slave-csg5 krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:35.171617 22218 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:12:35.172714 22218 webserver.cc:460] Webserver started at http://127.31.156.126:42191/ using document root <none> and password file <none>
I20260812 07:12:35.173379 22218 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:35.173487 22218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:35.173746 22218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:35.176167 22218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "7b7ce1f59db4405ba8b2c6cf21a43a7a"
format_stamp: "Formatted at 2026-08-12 07:12:35 on dist-test-slave-csg5"
server_key: "acd02731449543cf8a0c9210833b1edc"
server_key_iv: "60cd966ec48a4fd75cfc6dc4e0f21c17"
server_key_version: "encryptionkey@0"
I20260812 07:12:35.176812 22218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "7b7ce1f59db4405ba8b2c6cf21a43a7a"
format_stamp: "Formatted at 2026-08-12 07:12:35 on dist-test-slave-csg5"
server_key: "acd02731449543cf8a0c9210833b1edc"
server_key_iv: "60cd966ec48a4fd75cfc6dc4e0f21c17"
server_key_version: "encryptionkey@0"
I20260812 07:12:35.181174 22218 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:12:35.184392 22238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:35.186075 22218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:12:35.186353 22218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "7b7ce1f59db4405ba8b2c6cf21a43a7a"
format_stamp: "Formatted at 2026-08-12 07:12:35 on dist-test-slave-csg5"
server_key: "acd02731449543cf8a0c9210833b1edc"
server_key_iv: "60cd966ec48a4fd75cfc6dc4e0f21c17"
server_key_version: "encryptionkey@0"
I20260812 07:12:35.186518 22218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:35.199292 22218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:35.203078 22218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:35.203328 22218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:35.211817 22218 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:46517
I20260812 07:12:35.211854 22305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:46517 every 8 connection(s)
I20260812 07:12:35.213119 22218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:35.216581 22306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:35.221451 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 22218
I20260812 07:12:35.221658 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:35.221992 32369 external_mini_cluster.cc:1428] Setting key 86fa0d1b6ebf69e5a026b83aa91134f6
I20260812 07:12:35.223623 22306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a: Bootstrap starting.
I20260812 07:12:35.226899 22306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:35.227927 22306 log.cc:824] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:35.230432 22306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a: No bootstrap required, opened a new log
Aug 12 07:12:35 dist-test-slave-csg5 krb5kdc[22203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:12:35.235531 22306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ce1f59db4405ba8b2c6cf21a43a7a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46517 } }
I20260812 07:12:35.235918 22306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:35.235999 22306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b7ce1f59db4405ba8b2c6cf21a43a7a, State: Initialized, Role: FOLLOWER
I20260812 07:12:35.236641 22306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ce1f59db4405ba8b2c6cf21a43a7a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46517 } }
I20260812 07:12:35.236820 22306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:35.236886 22306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:35.236994 22306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:35.238365 22306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ce1f59db4405ba8b2c6cf21a43a7a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46517 } }
I20260812 07:12:35.238821 22306 leader_election.cc:302] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b7ce1f59db4405ba8b2c6cf21a43a7a; no voters:
I20260812 07:12:35.239244 22306 leader_election.cc:288] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:35.239418 22312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:35.239807 22312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [term 1 LEADER]: Becoming Leader. State: Replica: 7b7ce1f59db4405ba8b2c6cf21a43a7a, State: Running, Role: LEADER
I20260812 07:12:35.240201 22312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ce1f59db4405ba8b2c6cf21a43a7a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46517 } }
I20260812 07:12:35.240751 22306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:35.241636 22312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b7ce1f59db4405ba8b2c6cf21a43a7a. Latest consensus state: current_term: 1 leader_uuid: "7b7ce1f59db4405ba8b2c6cf21a43a7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ce1f59db4405ba8b2c6cf21a43a7a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46517 } } }
I20260812 07:12:35.241722 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:35.223962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54150 (local address 127.31.156.126:46517)
0812 07:12:35.224501 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:35.224529 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:35.224586 (+ 57us) server_negotiation.cc:407] Connection header received
0812 07:12:35.225621 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:35.225650 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:35.226149 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:35.226594 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:35.227678 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.229068 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.230390 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.231179 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.231366 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:35.234895 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:35.234904 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:35.234927 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:35.234973 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:35.237334 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:35.237893 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:35.237900 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:35.237907 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:35.238077 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:35.238388 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:35.238392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:35.238393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:35.239028 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:35.239251 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:35.240257 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0812 07:12:35.240610 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":163,"threads_started":1}
I20260812 07:12:35.241772 22312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:35.242223 22313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b7ce1f59db4405ba8b2c6cf21a43a7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7ce1f59db4405ba8b2c6cf21a43a7a" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46517 } } }
I20260812 07:12:35.242350 22313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:35.242975 22321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:35.246507 22321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:35.254036 22321 catalog_manager.cc:1347] Generated new cluster ID: c1f5597b55134b53bddc9654fd6b0162
I20260812 07:12:35.254158 22321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:35.285000 22321 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:35.286532 22321 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:35.308277 22321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a: Generated new TSK 0
I20260812 07:12:35.309262 22321 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.156.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
Entry for principal kudu/127.31.156.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab.
I20260812 07:12:35.348507 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46517
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:35.475297 22337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:35.475699 22337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:35.475788 22337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:35.480623 22337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:35.480782 22337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:35.480894 22337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:12:35.487423 22337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.65.keytab
--principal=kudu/127.31.156.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:35.488718 22337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:35.489862 22337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:12:35.501389 22337 server_base.cc:1034] running on GCE node
W20260812 07:12:35.501392 22344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:35.501415 22345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:35.501423 22347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:35.502321 22337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:35.503304 22337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:35.504532 22337 hybrid_clock.cc:648] HybridClock initialized: now 1786518755504512 us; error 38 us; skew 500 ppm
Aug 12 07:12:35 dist-test-slave-csg5 krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:35.508180 22337 init.cc:377] Logged in from keytab as kudu/127.31.156.65@KRBTEST.COM (short username kudu)
I20260812 07:12:35.509333 22337 webserver.cc:460] Webserver started at http://127.31.156.65:44985/ using document root <none> and password file <none>
I20260812 07:12:35.510015 22337 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:35.510090 22337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:35.510347 22337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:35.512770 22337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "965111785cee4244871bf74745d58e6d"
format_stamp: "Formatted at 2026-08-12 07:12:35 on dist-test-slave-csg5"
server_key: "111005de2e6b0c1bae28fd3bde691ef9"
server_key_iv: "5c834c5dc5a459fa7430ec5996013672"
server_key_version: "encryptionkey@0"
I20260812 07:12:35.513471 22337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "965111785cee4244871bf74745d58e6d"
format_stamp: "Formatted at 2026-08-12 07:12:35 on dist-test-slave-csg5"
server_key: "111005de2e6b0c1bae28fd3bde691ef9"
server_key_iv: "5c834c5dc5a459fa7430ec5996013672"
server_key_version: "encryptionkey@0"
I20260812 07:12:35.518101 22337 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:12:35.521311 22354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:35.523092 22337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:35.523273 22337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "965111785cee4244871bf74745d58e6d"
format_stamp: "Formatted at 2026-08-12 07:12:35 on dist-test-slave-csg5"
server_key: "111005de2e6b0c1bae28fd3bde691ef9"
server_key_iv: "5c834c5dc5a459fa7430ec5996013672"
server_key_version: "encryptionkey@0"
I20260812 07:12:35.523427 22337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:35.537919 22337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:35.541904 22337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:35.542141 22337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:35.542960 22337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:35.543892 22337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:35.543972 22337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:35.544047 22337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:35.544091 22337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:35.553967 22337 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:41141
I20260812 07:12:35.553996 22490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:41141 every 8 connection(s)
I20260812 07:12:35.555048 22337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:35.557979 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 22337
I20260812 07:12:35.558133 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:12:35.558580 32369 external_mini_cluster.cc:1428] Setting key 3b3a2ff4044126318402d711f44334d3
WARNING: no policy specified for kudu/127.31.156.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.66@KRBTEST.COM" created.
Aug 12 07:12:35 dist-test-slave-csg5 krb5kdc[22203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.65@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:12:35.582298 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:35.557128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:42413 (local address 127.31.156.126:46517)
0812 07:12:35.557274 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:35.557278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:35.558387 (+ 1109us) server_negotiation.cc:407] Connection header received
0812 07:12:35.560046 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:35.560049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:35.560099 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:35.560313 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:35.562127 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.562800 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.564122 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.564448 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.564538 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:35.576717 (+ 12179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:35.576726 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:35.576733 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:35.576784 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:35.578904 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:35.579804 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:35.579808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:35.579810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:35.579918 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:35.580445 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:35.580448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:35.580450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:35.580791 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:35.580944 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:35.581806 (+ 862us) server_negotiation.cc:299] Negotiation successful
0812 07:12:35.581999 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:12:35.583128 22493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:35.557568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46517 (local address 127.31.156.65:42413)
0812 07:12:35.558181 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:35.558247 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:35.559814 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:35.560477 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:35.560488 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:35.561073 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:35.561893 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:35.561917 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.563010 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.563013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:35.563972 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:35.563983 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.564642 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.564649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:35.564912 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:35.574110 (+ 9198us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:35.574144 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:35.576415 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:35.579094 (+ 2679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:35.579106 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:35.579132 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:35.579635 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:35.580038 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:35.580045 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:35.580050 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:35.580294 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:35.580964 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:35.580974 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:35.581446 (+ 472us) client_negotiation.cc:769] Sending connection context
0812 07:12:35.581731 (+ 285us) client_negotiation.cc:240] Negotiation successful
0812 07:12:35.582051 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":126,"threads_started":1}
I20260812 07:12:35.584723 22491 heartbeater.cc:347] Connected to a master server at 127.31.156.126:46517
I20260812 07:12:35.585000 22491 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:35.585572 22491 heartbeater.cc:511] Master 127.31.156.126:46517 requested a full tablet report, sending...
I20260812 07:12:35.587523 22261 ts_manager.cc:194] Registered new tserver with Master: 965111785cee4244871bf74745d58e6d (127.31.156.65:41141)
I20260812 07:12:35.588712 22261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.65@KRBTEST.COM'} at 127.31.156.65:42413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
Entry for principal kudu/127.31.156.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab.
I20260812 07:12:35.599887 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46517
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:35.726275 22497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:35.726611 22497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:35.726681 22497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:35.731757 22497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:35.731855 22497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:35.731922 22497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:12:35.737993 22497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.66.keytab
--principal=kudu/127.31.156.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:35.739254 22497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:35.740383 22497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:35.751647 22503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:35.751799 22497 server_base.cc:1034] running on GCE node
W20260812 07:12:35.751652 22504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:35.752067 22506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:35.752499 22497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:35.753264 22497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:35.754498 22497 hybrid_clock.cc:648] HybridClock initialized: now 1786518755754472 us; error 36 us; skew 500 ppm
Aug 12 07:12:35 dist-test-slave-csg5 krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:35.758054 22497 init.cc:377] Logged in from keytab as kudu/127.31.156.66@KRBTEST.COM (short username kudu)
I20260812 07:12:35.759320 22497 webserver.cc:460] Webserver started at http://127.31.156.66:36115/ using document root <none> and password file <none>
I20260812 07:12:35.759974 22497 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:35.760079 22497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:35.760350 22497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:35.762722 22497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "8e0251a0648c435db3008b0a8814bfe6"
format_stamp: "Formatted at 2026-08-12 07:12:35 on dist-test-slave-csg5"
server_key: "18dbfb0af648ae5362b40b8da80350c6"
server_key_iv: "b469710aaa782b05151b7da9e9fb3ea2"
server_key_version: "encryptionkey@0"
I20260812 07:12:35.763381 22497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "8e0251a0648c435db3008b0a8814bfe6"
format_stamp: "Formatted at 2026-08-12 07:12:35 on dist-test-slave-csg5"
server_key: "18dbfb0af648ae5362b40b8da80350c6"
server_key_iv: "b469710aaa782b05151b7da9e9fb3ea2"
server_key_version: "encryptionkey@0"
I20260812 07:12:35.767669 22497 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:35.770431 22513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:35.771958 22497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:35.772152 22497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "8e0251a0648c435db3008b0a8814bfe6"
format_stamp: "Formatted at 2026-08-12 07:12:35 on dist-test-slave-csg5"
server_key: "18dbfb0af648ae5362b40b8da80350c6"
server_key_iv: "b469710aaa782b05151b7da9e9fb3ea2"
server_key_version: "encryptionkey@0"
I20260812 07:12:35.772398 22497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:35.792378 22497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:35.796234 22497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:35.796478 22497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:35.797173 22497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:35.798228 22497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:35.798319 22497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:35.798410 22497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:35.798460 22497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:35.808528 22497 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:35825
I20260812 07:12:35.808704 22646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:35825 every 8 connection(s)
I20260812 07:12:35.809594 22497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:12:35.819103 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 22497
I20260812 07:12:35.819260 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:12:35.819576 32369 external_mini_cluster.cc:1428] Setting key 32f1d120dc628479489e21a782297aec
Aug 12 07:12:35 dist-test-slave-csg5 krb5kdc[22203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.66@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:12:35.826459 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:35.811980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:54101 (local address 127.31.156.126:46517)
0812 07:12:35.812144 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:35.812149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:35.813157 (+ 1008us) server_negotiation.cc:407] Connection header received
0812 07:12:35.814582 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:35.814585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:35.814638 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:35.814820 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:35.816307 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.816916 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.818178 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.818424 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.818521 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:35.821919 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:35.821922 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:35.821924 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:35.821951 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:35.823790 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:35.824468 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:35.824474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:35.824478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:35.824582 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:35.824973 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:35.824979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:35.824984 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:35.825285 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:35.825398 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:35.826029 (+ 631us) server_negotiation.cc:299] Negotiation successful
0812 07:12:35.826205 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:12:35.827489 22650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:35.812462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46517 (local address 127.31.156.66:54101)
0812 07:12:35.812986 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:35.813034 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:35.814346 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:35.815004 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:35.815011 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:35.815478 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:35.815996 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:35.816010 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.817054 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.817057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:35.818028 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:35.818038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.818586 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.818589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:35.818813 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:35.819561 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:35.819580 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:35.821673 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:35.823978 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:35.823985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:35.824001 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:35.824327 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:35.824685 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:35.824688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:35.824689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:35.824864 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:35.825415 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:35.825420 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:35.825708 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 07:12:35.825940 (+ 232us) client_negotiation.cc:240] Negotiation successful
0812 07:12:35.826264 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":128,"threads_started":1}
I20260812 07:12:35.829048 22647 heartbeater.cc:347] Connected to a master server at 127.31.156.126:46517
I20260812 07:12:35.829360 22647 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:35.830075 22647 heartbeater.cc:511] Master 127.31.156.126:46517 requested a full tablet report, sending...
I20260812 07:12:35.831550 22261 ts_manager.cc:194] Registered new tserver with Master: 8e0251a0648c435db3008b0a8814bfe6 (127.31.156.66:35825)
WARNING: no policy specified for kudu/127.31.156.67@KRBTEST.COM; defaulting to no policy
I20260812 07:12:35.832291 22261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.66@KRBTEST.COM'} at 127.31.156.66:54101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
Entry for principal kudu/127.31.156.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab.
I20260812 07:12:35.858767 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46517
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:41177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:35.985083 22653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:35.985455 22653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:35.985504 22653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:35.989817 22653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:35.989902 22653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:35.989966 22653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:12:35.996202 22653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:41177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.67.keytab
--principal=kudu/127.31.156.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:46517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:35.997502 22653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:35.998587 22653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:36.010680 22664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:36.010669 22660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:36.010648 22662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:36.010735 22653 server_base.cc:1034] running on GCE node
I20260812 07:12:36.011444 22653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:36.012271 22653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:36.013490 22653 hybrid_clock.cc:648] HybridClock initialized: now 1786518756013471 us; error 38 us; skew 500 ppm
Aug 12 07:12:36 dist-test-slave-csg5 krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518756, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:36.016927 22653 init.cc:377] Logged in from keytab as kudu/127.31.156.67@KRBTEST.COM (short username kudu)
I20260812 07:12:36.017952 22653 webserver.cc:460] Webserver started at http://127.31.156.67:42505/ using document root <none> and password file <none>
I20260812 07:12:36.018522 22653 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:36.018584 22653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:36.018842 22653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:36.021119 22653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "5f6a098aee5b436dbb354aad952eb563"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "dbd4b590e7e483845e10d833def2077d"
server_key_iv: "0956019f7732f79725daab6b6fcf18ad"
server_key_version: "encryptionkey@0"
I20260812 07:12:36.021761 22653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "5f6a098aee5b436dbb354aad952eb563"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "dbd4b590e7e483845e10d833def2077d"
server_key_iv: "0956019f7732f79725daab6b6fcf18ad"
server_key_version: "encryptionkey@0"
I20260812 07:12:36.026037 22653 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:12:36.029270 22671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:36.030961 22653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:36.031142 22653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "5f6a098aee5b436dbb354aad952eb563"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "dbd4b590e7e483845e10d833def2077d"
server_key_iv: "0956019f7732f79725daab6b6fcf18ad"
server_key_version: "encryptionkey@0"
I20260812 07:12:36.031296 22653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:36.058298 22653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:36.062000 22653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:36.062342 22653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:36.063149 22653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:36.064126 22653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:36.064205 22653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:36.064288 22653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:36.064332 22653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:36.075525 22653 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:40263
I20260812 07:12:36.075577 22796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:40263 every 8 connection(s)
I20260812 07:12:36.076565 22653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:36.078027 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 22653
I20260812 07:12:36.078178 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:12:36.078544 32369 external_mini_cluster.cc:1428] Setting key f1fe9fbacdcea9ae743af219f4d82d57
Aug 12 07:12:36 dist-test-slave-csg5 krb5kdc[22203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518756, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.67@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:12:36.094967 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.079461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:33001 (local address 127.31.156.126:46517)
0812 07:12:36.079631 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.079635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.080704 (+ 1069us) server_negotiation.cc:407] Connection header received
0812 07:12:36.082384 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.082388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.082447 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.082527 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:36.083981 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.084570 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.085821 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.086058 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.086196 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.089497 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:36.089501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:36.089505 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:36.089544 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:36.092008 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.092757 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.092763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.092765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.092889 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.093363 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.093367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.093369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.093645 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:36.093729 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.094546 (+ 817us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.094724 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:12:36.096760 22799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.079845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:46517 (local address 127.31.156.67:33001)
0812 07:12:36.080536 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.080587 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.082152 (+ 1565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.082754 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.082762 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.083179 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:36.083805 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.083835 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.084718 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.084722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.085678 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.085688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.086248 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.086252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.086438 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.087125 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:36.087143 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:36.089265 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:36.092187 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:36.092202 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:36.092234 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:36.092605 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:36.093017 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:36.093021 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:36.093022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:36.093245 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:36.093834 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:36.093843 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:36.094235 (+ 392us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.095219 (+ 984us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.095701 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":191,"threads_started":1}
I20260812 07:12:36.099203 22797 heartbeater.cc:347] Connected to a master server at 127.31.156.126:46517
I20260812 07:12:36.099534 22797 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:36.100180 22797 heartbeater.cc:511] Master 127.31.156.126:46517 requested a full tablet report, sending...
I20260812 07:12:36.101722 22261 ts_manager.cc:194] Registered new tserver with Master: 5f6a098aee5b436dbb354aad952eb563 (127.31.156.67:40263)
I20260812 07:12:36.102553 22261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.156.67@KRBTEST.COM'} at 127.31.156.67:33001
I20260812 07:12:36.107055 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:12:36.118645 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.109537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54154 (local address 127.31.156.126:46517)
0812 07:12:36.109713 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.109719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.109861 (+ 142us) server_negotiation.cc:407] Connection header received
0812 07:12:36.110038 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.110041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.110085 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.110171 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:36.111058 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.111599 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.112497 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.112721 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.112843 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.114626 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:36.114630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:36.114632 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:36.114727 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:36.116665 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.117219 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.117223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.117224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.117329 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.117629 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.117632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.117634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.117897 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:36.118050 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.118319 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.118456 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:12:36.122596 22261 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:12:36.126300 22261 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:12:36.145184 22810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.137944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58704 (local address 127.31.156.65:41141)
0812 07:12:36.138504 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.138512 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.139346 (+ 834us) server_negotiation.cc:407] Connection header received
0812 07:12:36.139568 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.139573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.139746 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.139883 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:36.140303 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.141463 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.143536 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.144299 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.144398 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.144526 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.144856 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.144985 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":133,"threads_started":1}
I20260812 07:12:36.145184 22815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.138770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37082 (local address 127.31.156.66:35825)
0812 07:12:36.139564 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.139581 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.139598 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:12:36.139665 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.139669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.139871 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.140062 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:36.141251 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.142454 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.143829 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.144525 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.144698 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.144872 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.144918 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.144996 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":154,"threads_started":1}
I20260812 07:12:36.145228 22812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.138122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:35825 (local address 127.0.0.1:37082)
0812 07:12:36.139209 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.139224 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.139311 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.140033 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.140036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.140048 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:36.140166 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.140173 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.142605 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.142611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.143688 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.143699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.144673 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.144677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.144790 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.144815 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.144881 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.145046 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"mutex_wait_us":21,"thread_start_us":104,"threads_started":1}
I20260812 07:12:36.146121 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.137799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:41141 (local address 127.0.0.1:58704)
0812 07:12:36.139260 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.139272 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.139414 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.139976 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.139980 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.140000 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:36.140168 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.140173 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.141624 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.141637 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.143392 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.143405 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.144443 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.144449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.144593 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.144653 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.145893 (+ 1240us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.145983 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":134,"thread_start_us":120,"threads_started":1}
I20260812 07:12:36.146350 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.138102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:40263 (local address 127.0.0.1:36984)
0812 07:12:36.138876 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.138949 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.139087 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.139779 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.139785 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.139810 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:36.139994 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.140006 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.141624 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.141628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.143392 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.143405 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.145901 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.145907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.146008 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.146035 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.146107 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.146200 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":409,"thread_start_us":134,"threads_started":1}
I20260812 07:12:36.146953 22813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.138696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36984 (local address 127.31.156.67:40263)
0812 07:12:36.139230 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.139243 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.139268 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:12:36.139361 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.139369 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.139605 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.139774 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:36.140144 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.141463 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.144625 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.145573 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.145684 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.145830 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.146693 (+ 863us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.146810 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":147,"threads_started":1}
I20260812 07:12:36.148926 22568 tablet_service.cc:1467] Processing CreateTablet for tablet e4f4b7c7fd044101a2358506a5a2ad53 (DEFAULT_TABLE table=test-table [id=f9d6951af1ed449cb319f50de6a6a0ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:36.149395 22721 tablet_service.cc:1467] Processing CreateTablet for tablet e4f4b7c7fd044101a2358506a5a2ad53 (DEFAULT_TABLE table=test-table [id=f9d6951af1ed449cb319f50de6a6a0ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:36.149456 22408 tablet_service.cc:1467] Processing CreateTablet for tablet e4f4b7c7fd044101a2358506a5a2ad53 (DEFAULT_TABLE table=test-table [id=f9d6951af1ed449cb319f50de6a6a0ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:36.150209 22568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f4b7c7fd044101a2358506a5a2ad53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:36.150436 22408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f4b7c7fd044101a2358506a5a2ad53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:36.150462 22721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f4b7c7fd044101a2358506a5a2ad53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:36.157327 22816 tablet_bootstrap.cc:492] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563: Bootstrap starting.
I20260812 07:12:36.158936 22817 tablet_bootstrap.cc:492] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d: Bootstrap starting.
I20260812 07:12:36.159346 22818 tablet_bootstrap.cc:492] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6: Bootstrap starting.
I20260812 07:12:36.160228 22816 tablet_bootstrap.cc:654] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:36.161365 22817 tablet_bootstrap.cc:654] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:36.161790 22818 tablet_bootstrap.cc:654] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:36.162071 22816 log.cc:824] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:36.162439 22817 log.cc:824] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:36.163080 22818 log.cc:824] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:36.164958 22816 tablet_bootstrap.cc:492] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563: No bootstrap required, opened a new log
I20260812 07:12:36.165546 22816 ts_tablet_manager.cc:1397] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:12:36.165720 22818 tablet_bootstrap.cc:492] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6: No bootstrap required, opened a new log
I20260812 07:12:36.165930 22818 ts_tablet_manager.cc:1397] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 07:12:36.168561 22817 tablet_bootstrap.cc:492] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d: No bootstrap required, opened a new log
I20260812 07:12:36.168818 22817 ts_tablet_manager.cc:1397] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260812 07:12:36.170325 22818 raft_consensus.cc:348] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965111785cee4244871bf74745d58e6d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41141 } } peers { permanent_uuid: "8e0251a0648c435db3008b0a8814bfe6" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 35825 } } peers { permanent_uuid: "5f6a098aee5b436dbb354aad952eb563" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40263 } }
I20260812 07:12:36.170325 22816 raft_consensus.cc:348] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965111785cee4244871bf74745d58e6d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41141 } } peers { permanent_uuid: "8e0251a0648c435db3008b0a8814bfe6" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 35825 } } peers { permanent_uuid: "5f6a098aee5b436dbb354aad952eb563" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40263 } }
I20260812 07:12:36.170696 22818 raft_consensus.cc:374] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:36.170688 22816 raft_consensus.cc:374] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:36.170751 22816 raft_consensus.cc:729] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f6a098aee5b436dbb354aad952eb563, State: Initialized, Role: FOLLOWER
I20260812 07:12:36.170763 22818 raft_consensus.cc:729] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e0251a0648c435db3008b0a8814bfe6, State: Initialized, Role: FOLLOWER
I20260812 07:12:36.171295 22818 consensus_queue.cc:260] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965111785cee4244871bf74745d58e6d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41141 } } peers { permanent_uuid: "8e0251a0648c435db3008b0a8814bfe6" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 35825 } } peers { permanent_uuid: "5f6a098aee5b436dbb354aad952eb563" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40263 } }
I20260812 07:12:36.171295 22816 consensus_queue.cc:260] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965111785cee4244871bf74745d58e6d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41141 } } peers { permanent_uuid: "8e0251a0648c435db3008b0a8814bfe6" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 35825 } } peers { permanent_uuid: "5f6a098aee5b436dbb354aad952eb563" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40263 } }
I20260812 07:12:36.172159 22797 heartbeater.cc:503] Master 127.31.156.126:46517 was elected leader, sending a full tablet report...
I20260812 07:12:36.172500 22647 heartbeater.cc:503] Master 127.31.156.126:46517 was elected leader, sending a full tablet report...
I20260812 07:12:36.172535 22818 ts_tablet_manager.cc:1428] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 07:12:36.173655 22816 ts_tablet_manager.cc:1428] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260812 07:12:36.173893 22823 raft_consensus.cc:482] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:36.174095 22823 raft_consensus.cc:504] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965111785cee4244871bf74745d58e6d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41141 } } peers { permanent_uuid: "8e0251a0648c435db3008b0a8814bfe6" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 35825 } } peers { permanent_uuid: "5f6a098aee5b436dbb354aad952eb563" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40263 } }
I20260812 07:12:36.174728 22817 raft_consensus.cc:348] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965111785cee4244871bf74745d58e6d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41141 } } peers { permanent_uuid: "8e0251a0648c435db3008b0a8814bfe6" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 35825 } } peers { permanent_uuid: "5f6a098aee5b436dbb354aad952eb563" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40263 } }
I20260812 07:12:36.175082 22817 raft_consensus.cc:374] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:36.175153 22817 raft_consensus.cc:729] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965111785cee4244871bf74745d58e6d, State: Initialized, Role: FOLLOWER
I20260812 07:12:36.175662 22817 consensus_queue.cc:260] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965111785cee4244871bf74745d58e6d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41141 } } peers { permanent_uuid: "8e0251a0648c435db3008b0a8814bfe6" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 35825 } } peers { permanent_uuid: "5f6a098aee5b436dbb354aad952eb563" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40263 } }
I20260812 07:12:36.176487 22823 leader_election.cc:288] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 965111785cee4244871bf74745d58e6d (127.31.156.65:41141), 5f6a098aee5b436dbb354aad952eb563 (127.31.156.67:40263)
I20260812 07:12:36.177351 22817 ts_tablet_manager.cc:1428] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 07:12:36.177726 22491 heartbeater.cc:503] Master 127.31.156.126:46517 was elected leader, sending a full tablet report...
I20260812 07:12:36.181646 22810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.175992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:38297 (local address 127.31.156.65:41141)
0812 07:12:36.176128 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.176135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.176156 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:12:36.176339 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.176345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.176406 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.176516 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:36.176870 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.177702 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.180371 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.181034 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.181165 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.181217 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.181467 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.181540 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 07:12:36.181661 22650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.175801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:41141 (local address 127.31.156.66:38297)
0812 07:12:36.176086 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.176102 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.176230 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.176545 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.176548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.176569 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:36.176733 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.176742 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.178912 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.178916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.180155 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.180164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.181170 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.181174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.181288 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.181351 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.181433 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.181519 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260812 07:12:36.182700 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.176232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:40263 (local address 127.31.156.66:33727)
0812 07:12:36.177819 (+ 1587us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.177835 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.177976 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.178435 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.178437 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.178454 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:36.178583 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.178589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.180030 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.180033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.181238 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.181250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.182117 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.182120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.182313 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.182356 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.182453 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.182528 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1510,"thread_start_us":114,"threads_started":1}
I20260812 07:12:36.182719 22813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.176342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:33727 (local address 127.31.156.67:40263)
0812 07:12:36.176592 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.176597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.177910 (+ 1313us) server_negotiation.cc:407] Connection header received
0812 07:12:36.178123 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.178126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.178195 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.178264 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:36.179001 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.179766 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.181386 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.181989 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.182136 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.182187 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.182485 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.182561 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260812 07:12:36.182996 22436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f4b7c7fd044101a2358506a5a2ad53" candidate_uuid: "8e0251a0648c435db3008b0a8814bfe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "965111785cee4244871bf74745d58e6d" is_pre_election: true
I20260812 07:12:36.183327 22436 raft_consensus.cc:2454] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e0251a0648c435db3008b0a8814bfe6 in term 0.
I20260812 07:12:36.183405 22743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f4b7c7fd044101a2358506a5a2ad53" candidate_uuid: "8e0251a0648c435db3008b0a8814bfe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6a098aee5b436dbb354aad952eb563" is_pre_election: true
I20260812 07:12:36.183703 22743 raft_consensus.cc:2454] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e0251a0648c435db3008b0a8814bfe6 in term 0.
I20260812 07:12:36.183912 22515 leader_election.cc:302] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e0251a0648c435db3008b0a8814bfe6, 965111785cee4244871bf74745d58e6d; no voters:
I20260812 07:12:36.184242 22823 raft_consensus.cc:2781] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:12:36.184348 22823 raft_consensus.cc:482] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:12:36.184407 22823 raft_consensus.cc:3037] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:36.185555 22823 raft_consensus.cc:504] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965111785cee4244871bf74745d58e6d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41141 } } peers { permanent_uuid: "8e0251a0648c435db3008b0a8814bfe6" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 35825 } } peers { permanent_uuid: "5f6a098aee5b436dbb354aad952eb563" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40263 } }
I20260812 07:12:36.185976 22823 leader_election.cc:288] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [CANDIDATE]: Term 1 election: Requested vote from peers 965111785cee4244871bf74745d58e6d (127.31.156.65:41141), 5f6a098aee5b436dbb354aad952eb563 (127.31.156.67:40263)
I20260812 07:12:36.186448 22743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f4b7c7fd044101a2358506a5a2ad53" candidate_uuid: "8e0251a0648c435db3008b0a8814bfe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6a098aee5b436dbb354aad952eb563"
I20260812 07:12:36.186491 22436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f4b7c7fd044101a2358506a5a2ad53" candidate_uuid: "8e0251a0648c435db3008b0a8814bfe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "965111785cee4244871bf74745d58e6d"
I20260812 07:12:36.186609 22743 raft_consensus.cc:3037] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:36.186640 22436 raft_consensus.cc:3037] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:36.188120 22436 raft_consensus.cc:2454] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e0251a0648c435db3008b0a8814bfe6 in term 1.
I20260812 07:12:36.188120 22743 raft_consensus.cc:2454] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e0251a0648c435db3008b0a8814bfe6 in term 1.
I20260812 07:12:36.188602 22518 leader_election.cc:302] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f6a098aee5b436dbb354aad952eb563, 8e0251a0648c435db3008b0a8814bfe6; no voters:
I20260812 07:12:36.188896 22823 raft_consensus.cc:2781] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:36.189213 22823 raft_consensus.cc:686] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 1 LEADER]: Becoming Leader. State: Replica: 8e0251a0648c435db3008b0a8814bfe6, State: Running, Role: LEADER
I20260812 07:12:36.189638 22823 consensus_queue.cc:237] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965111785cee4244871bf74745d58e6d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41141 } } peers { permanent_uuid: "8e0251a0648c435db3008b0a8814bfe6" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 35825 } } peers { permanent_uuid: "5f6a098aee5b436dbb354aad952eb563" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40263 } }
I20260812 07:12:36.193554 22261 catalog_manager.cc:5495] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e0251a0648c435db3008b0a8814bfe6 (127.31.156.66). New cstate: current_term: 1 leader_uuid: "8e0251a0648c435db3008b0a8814bfe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965111785cee4244871bf74745d58e6d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e0251a0648c435db3008b0a8814bfe6" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 35825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f6a098aee5b436dbb354aad952eb563" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40263 } health_report { overall_health: UNKNOWN } } }
I20260812 07:12:36.214407 22815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.210060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37088 (local address 127.31.156.66:35825)
0812 07:12:36.210205 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.210210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.210227 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:12:36.210357 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.210359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.210420 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.210540 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:12:36.210892 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.211586 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.212549 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.212785 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.212906 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.213262 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:12:36.213827 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:12:36.214017 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.214086 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.214160 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:12:36.221750 22436 raft_consensus.cc:1261] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d [term 1 FOLLOWER]: Refusing update from remote peer 8e0251a0648c435db3008b0a8814bfe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:36.221762 22743 raft_consensus.cc:1261] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563 [term 1 FOLLOWER]: Refusing update from remote peer 8e0251a0648c435db3008b0a8814bfe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:36.222759 22823 consensus_queue.cc:1035] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965111785cee4244871bf74745d58e6d" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 41141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:36.223131 22828 consensus_queue.cc:1035] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f6a098aee5b436dbb354aad952eb563" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 40263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:36.233743 22832 mvcc.cc:205] Tried to move back new op lower bound from 7317580825475129344 to 7317580825356881920. Current Snapshot: MvccSnapshot[applied={T|T < 7317580825475129344}]
I20260812 07:12:36.238835 22834 mvcc.cc:205] Tried to move back new op lower bound from 7317580825475129344 to 7317580825356881920. Current Snapshot: MvccSnapshot[applied={T|T < 7317580825475129344}]
I20260812 07:12:36.239266 22833 mvcc.cc:205] Tried to move back new op lower bound from 7317580825475129344 to 7317580825356881920. Current Snapshot: MvccSnapshot[applied={T|T < 7317580825475129344}]
I20260812 07:12:36.249930 22261 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:36.250177 22261 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:36.253242 22261 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b7ce1f59db4405ba8b2c6cf21a43a7a: Sending DeleteTablet for 3 replicas of tablet e4f4b7c7fd044101a2358506a5a2ad53
I20260812 07:12:36.254397 22568 tablet_service.cc:1514] Processing DeleteTablet for tablet e4f4b7c7fd044101a2358506a5a2ad53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:36 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:37082
I20260812 07:12:36.254379 22408 tablet_service.cc:1514] Processing DeleteTablet for tablet e4f4b7c7fd044101a2358506a5a2ad53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:36 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:58704
I20260812 07:12:36.255211 22845 tablet_replica.cc:330] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d: stopping tablet replica
I20260812 07:12:36.255182 22721 tablet_service.cc:1514] Processing DeleteTablet for tablet e4f4b7c7fd044101a2358506a5a2ad53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:36 UTC) from {username='kudu', principal='kudu/127.31.156.126@KRBTEST.COM'} at 127.0.0.1:36984
I20260812 07:12:36.255820 22845 raft_consensus.cc:2229] T e4f4b7c7fd044101a2358506a5a2ad53 P 965111785cee4244871bf74745d58e6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:36.255966 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 22337
I20260812 07:12:36.255955 22844 tablet_replica.cc:330] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6: stopping tablet replica
I20260812 07:12:36.259456 22846 tablet_replica.cc:330] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563: stopping tablet replica
I20260812 07:12:36.260021 22846 raft_consensus.cc:2229] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:36.260433 22846 raft_consensus.cc:2258] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:36.260982 22844 raft_consensus.cc:2229] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:12:36.261556 22844 raft_consensus.cc:2258] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:36.263607 22844 ts_tablet_manager.cc:1905] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:36.265005 22846 ts_tablet_manager.cc:1905] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:36.267132 22844 ts_tablet_manager.cc:1918] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:36.267288 22844 log.cc:1196] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/e4f4b7c7fd044101a2358506a5a2ad53
I20260812 07:12:36.267678 22844 ts_tablet_manager.cc:1939] T e4f4b7c7fd044101a2358506a5a2ad53 P 8e0251a0648c435db3008b0a8814bfe6: Deleting consensus metadata
I20260812 07:12:36.268786 22846 ts_tablet_manager.cc:1918] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:36.268988 22846 log.cc:1196] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/e4f4b7c7fd044101a2358506a5a2ad53
I20260812 07:12:36.269454 22846 ts_tablet_manager.cc:1939] T e4f4b7c7fd044101a2358506a5a2ad53 P 5f6a098aee5b436dbb354aad952eb563: Deleting consensus metadata
I20260812 07:12:36.269464 22240 catalog_manager.cc:4841] TS 8e0251a0648c435db3008b0a8814bfe6 (127.31.156.66:35825): tablet e4f4b7c7fd044101a2358506a5a2ad53 (table test-table [id=f9d6951af1ed449cb319f50de6a6a0ee]) successfully deleted
W20260812 07:12:36.270114 22239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:41141 (error 108)
I20260812 07:12:36.270454 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 22497
I20260812 07:12:36.273420 22243 catalog_manager.cc:4841] TS 5f6a098aee5b436dbb354aad952eb563 (127.31.156.67:40263): tablet e4f4b7c7fd044101a2358506a5a2ad53 (table test-table [id=f9d6951af1ed449cb319f50de6a6a0ee]) successfully deleted
I20260812 07:12:36.275058 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.274560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:41141 (local address 127.0.0.1:58716)
0812 07:12:36.274835 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.274963 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:41141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
W20260812 07:12:36.275300 22239 catalog_manager.cc:4584] TS 965111785cee4244871bf74745d58e6d (127.31.156.65:41141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4f4b7c7fd044101a2358506a5a2ad53: Network error: Client connection negotiation failed: client connection to 127.31.156.65:41141: connect: Connection refused (error 111)
I20260812 07:12:36.281299 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 22653
I20260812 07:12:36.292801 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 22218
2026-08-12T07:12:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2628 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T07:12:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:36Z Disabled control of system clock
I20260812 07:12:36.336175 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:39843
--webserver_interface=127.31.156.126
--webserver_port=0
--builtin_ntp_servers=127.31.156.84:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:39843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:12:36.470909 22852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:36.471223 22852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:36.471268 22852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:36.475811 22852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:36.475894 22852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:36.475911 22852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:36.475929 22852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:36.475945 22852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:36.481148 22852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:39843
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:39843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:36.482385 22852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:36.484012 22852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:36.493726 22862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:36.493716 22859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:36.493747 22860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:36.494249 22852 server_base.cc:1034] running on GCE node
I20260812 07:12:36.495005 22852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:36.496297 22852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:36.497484 22852 hybrid_clock.cc:648] HybridClock initialized: now 1786518756497438 us; error 66 us; skew 500 ppm
I20260812 07:12:36.499716 22852 webserver.cc:460] Webserver started at http://127.31.156.126:36363/ using document root <none> and password file <none>
I20260812 07:12:36.500324 22852 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:36.500396 22852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:36.500584 22852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:36.503015 22852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "86aaa9ff64734c7683e669f0a2815e0e"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "741d979c70f9a6adcc9e8c507772f71f"
server_key_iv: "967586e831b2571670ac1737e4387bb8"
server_key_version: "encryptionkey@0"
I20260812 07:12:36.503613 22852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "86aaa9ff64734c7683e669f0a2815e0e"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "741d979c70f9a6adcc9e8c507772f71f"
server_key_iv: "967586e831b2571670ac1737e4387bb8"
server_key_version: "encryptionkey@0"
I20260812 07:12:36.508026 22852 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 07:12:36.511060 22870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:36.512595 22852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 07:12:36.512782 22852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "86aaa9ff64734c7683e669f0a2815e0e"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "741d979c70f9a6adcc9e8c507772f71f"
server_key_iv: "967586e831b2571670ac1737e4387bb8"
server_key_version: "encryptionkey@0"
I20260812 07:12:36.512926 22852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:36.536182 22852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:36.537010 22852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:36.537241 22852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:36.545805 22852 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:39843
I20260812 07:12:36.545809 22930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:39843 every 8 connection(s)
I20260812 07:12:36.546986 22852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:36.550474 22931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:36.555356 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 22852
I20260812 07:12:36.555488 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:36.555817 32369 external_mini_cluster.cc:1428] Setting key 5e37bdb65ad38c87e6b4a67a5d58dd35
I20260812 07:12:36.557493 22931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e: Bootstrap starting.
I20260812 07:12:36.560570 22931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:36.561599 22931 log.cc:824] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:36.564234 22931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e: No bootstrap required, opened a new log
I20260812 07:12:36.567350 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.557791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33142 (local address 127.31.156.126:39843)
0812 07:12:36.558561 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.558583 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.558632 (+ 49us) server_negotiation.cc:407] Connection header received
0812 07:12:36.559628 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.559653 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.560205 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.560627 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:36.561168 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.562494 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.563500 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.564037 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.564238 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.565275 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:36.565281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:36.565296 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:36.565736 (+ 440us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:36.565803 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:36.565825 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:36.565947 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.566114 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.566429 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":142,"threads_started":1}
I20260812 07:12:36.568904 22931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86aaa9ff64734c7683e669f0a2815e0e" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39843 } }
I20260812 07:12:36.569254 22931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:36.569314 22931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86aaa9ff64734c7683e669f0a2815e0e, State: Initialized, Role: FOLLOWER
I20260812 07:12:36.569849 22931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86aaa9ff64734c7683e669f0a2815e0e" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39843 } }
I20260812 07:12:36.570073 22931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:36.570153 22931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:36.570293 22931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:36.571918 22931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86aaa9ff64734c7683e669f0a2815e0e" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39843 } }
I20260812 07:12:36.572296 22931 leader_election.cc:302] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86aaa9ff64734c7683e669f0a2815e0e; no voters:
I20260812 07:12:36.572606 22931 leader_election.cc:288] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:36.572898 22939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:36.573153 22939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [term 1 LEADER]: Becoming Leader. State: Replica: 86aaa9ff64734c7683e669f0a2815e0e, State: Running, Role: LEADER
I20260812 07:12:36.573587 22939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86aaa9ff64734c7683e669f0a2815e0e" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39843 } }
I20260812 07:12:36.574035 22931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:36.575506 22940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86aaa9ff64734c7683e669f0a2815e0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86aaa9ff64734c7683e669f0a2815e0e" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39843 } } }
I20260812 07:12:36.575665 22940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:36.576104 22950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:36.576377 22941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86aaa9ff64734c7683e669f0a2815e0e. Latest consensus state: current_term: 1 leader_uuid: "86aaa9ff64734c7683e669f0a2815e0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86aaa9ff64734c7683e669f0a2815e0e" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 39843 } } }
I20260812 07:12:36.576502 22941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:36.579308 22950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:36.586784 22950 catalog_manager.cc:1347] Generated new cluster ID: 9b223646d8f14928a9a812db82706c7d
I20260812 07:12:36.586957 22950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:36.602784 22950 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:36.604015 22950 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:36.610474 22950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e: Generated new TSK 0
I20260812 07:12:36.611433 22950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:36.616668 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.65:0
--local_ip_for_outbound_sockets=127.31.156.65
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39843
--builtin_ntp_servers=127.31.156.84:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:12:36.748306 22960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:36.748643 22960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:36.748687 22960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:36.753059 22960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:36.753152 22960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:36.753213 22960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.65
I20260812 07:12:36.758584 22960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.156.65
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.65
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:36.759922 22960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:36.761451 22960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:12:36.772822 22960 server_base.cc:1034] running on GCE node
W20260812 07:12:36.772789 22965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:36.773031 22969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:36.772789 22967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:36.773584 22960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:36.774396 22960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:36.775612 22960 hybrid_clock.cc:648] HybridClock initialized: now 1786518756775588 us; error 67 us; skew 500 ppm
I20260812 07:12:36.777865 22960 webserver.cc:460] Webserver started at http://127.31.156.65:38297/ using document root <none> and password file <none>
I20260812 07:12:36.778532 22960 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:36.778640 22960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:36.778916 22960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:36.781255 22960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/data/instance:
uuid: "9e5bfde8100e4636a759cff05140891e"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "377a36e56d9da904ea0cdb1c554bc9ef"
server_key_iv: "b4df7998d38c42174700441f74e4c754"
server_key_version: "encryptionkey@0"
I20260812 07:12:36.781924 22960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance:
uuid: "9e5bfde8100e4636a759cff05140891e"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "377a36e56d9da904ea0cdb1c554bc9ef"
server_key_iv: "b4df7998d38c42174700441f74e4c754"
server_key_version: "encryptionkey@0"
I20260812 07:12:36.786444 22960 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 07:12:36.789585 22977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:36.791309 22960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:36.791495 22960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/wal
uuid: "9e5bfde8100e4636a759cff05140891e"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "377a36e56d9da904ea0cdb1c554bc9ef"
server_key_iv: "b4df7998d38c42174700441f74e4c754"
server_key_version: "encryptionkey@0"
I20260812 07:12:36.791647 22960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:36.812932 22960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:36.813784 22960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:36.814023 22960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:36.814740 22960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:36.815845 22960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:36.815927 22960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:36.816007 22960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:36.816056 22960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:36.826627 22960 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.65:35099
I20260812 07:12:36.826673 23105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.65:35099 every 8 connection(s)
I20260812 07:12:36.827767 22960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:36.836109 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 22960
I20260812 07:12:36.836287 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-0/wal/instance
I20260812 07:12:36.836613 32369 external_mini_cluster.cc:1428] Setting key 1d501ccf47b7832ec026f1367f61e3c5
I20260812 07:12:36.839169 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.66:0
--local_ip_for_outbound_sockets=127.31.156.66
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39843
--builtin_ntp_servers=127.31.156.84:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:12:36.839617 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.830178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.65:57257 (local address 127.31.156.126:39843)
0812 07:12:36.830341 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.830345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.831590 (+ 1245us) server_negotiation.cc:407] Connection header received
0812 07:12:36.833047 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.833051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.833103 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.833188 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:36.834186 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.834940 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.836200 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.836588 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.836716 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.838605 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:36.838611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:36.838614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:36.838632 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:36.838713 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:36.838731 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:36.838844 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.839043 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.839122 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:12:36.840480 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.830512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:39843 (local address 127.31.156.65:57257)
0812 07:12:36.831388 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.831437 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.832797 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.833372 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.833379 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.833467 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:36.834002 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.834016 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.835076 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.835079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.836045 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.836057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.836776 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.836781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.836986 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.837953 (+ 967us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:12:36.837955 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:36.838014 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:12:36.838105 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:36.838860 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:36.838875 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:36.838882 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.839045 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.839429 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":251,"threads_started":1}
I20260812 07:12:36.842024 23106 heartbeater.cc:347] Connected to a master server at 127.31.156.126:39843
I20260812 07:12:36.842418 23106 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:36.843111 23106 heartbeater.cc:511] Master 127.31.156.126:39843 requested a full tablet report, sending...
I20260812 07:12:36.844928 22888 ts_manager.cc:194] Registered new tserver with Master: 9e5bfde8100e4636a759cff05140891e (127.31.156.65:35099)
I20260812 07:12:36.846813 22888 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.65:57257
W20260812 07:12:36.969794 23110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:36.970099 23110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:36.970142 23110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:36.974896 23110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:36.974978 23110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:36.975041 23110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.66
I20260812 07:12:36.980473 23110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.156.66
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.66
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:36.981644 23110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:36.983088 23110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:36.993983 23116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:36.994127 23110 server_base.cc:1034] running on GCE node
W20260812 07:12:36.993981 23119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:36.994007 23117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:36.994794 23110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:36.995618 23110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:36.996817 23110 hybrid_clock.cc:648] HybridClock initialized: now 1786518756996783 us; error 63 us; skew 500 ppm
I20260812 07:12:36.999155 23110 webserver.cc:460] Webserver started at http://127.31.156.66:34155/ using document root <none> and password file <none>
I20260812 07:12:36.999814 23110 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:36.999918 23110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:37.000195 23110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:37.002522 23110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/data/instance:
uuid: "c48a3f53acb74e568a2a45c4b9c4b001"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "6742c7c3fa5d71e8ec2965a60584d433"
server_key_iv: "8d04787ce5a28b3f821354c1a0177f81"
server_key_version: "encryptionkey@0"
I20260812 07:12:37.003227 23110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance:
uuid: "c48a3f53acb74e568a2a45c4b9c4b001"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "6742c7c3fa5d71e8ec2965a60584d433"
server_key_iv: "8d04787ce5a28b3f821354c1a0177f81"
server_key_version: "encryptionkey@0"
I20260812 07:12:37.007534 23110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:37.010222 23125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:37.011934 23110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:37.012055 23110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/wal
uuid: "c48a3f53acb74e568a2a45c4b9c4b001"
format_stamp: "Formatted at 2026-08-12 07:12:36 on dist-test-slave-csg5"
server_key: "6742c7c3fa5d71e8ec2965a60584d433"
server_key_iv: "8d04787ce5a28b3f821354c1a0177f81"
server_key_version: "encryptionkey@0"
I20260812 07:12:37.012248 23110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:37.024574 23110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:37.025378 23110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:37.025611 23110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:37.026265 23110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:37.027386 23110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:37.027436 23110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:37.027477 23110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:37.027531 23110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:37.038194 23110 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.66:41919
I20260812 07:12:37.038296 23260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.66:41919 every 8 connection(s)
I20260812 07:12:37.039501 23110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/data/info.pb
I20260812 07:12:37.039971 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 23110
I20260812 07:12:37.040107 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/wal/instance
I20260812 07:12:37.040467 32369 external_mini_cluster.cc:1428] Setting key 4d68ede9d0775bc2c6034f8c2faefe19
I20260812 07:12:37.042943 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.156.67:0
--local_ip_for_outbound_sockets=127.31.156.67
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39843
--builtin_ntp_servers=127.31.156.84:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:12:37.052924 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.041805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.66:58167 (local address 127.31.156.126:39843)
0812 07:12:37.041951 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:37.041956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:37.043488 (+ 1532us) server_negotiation.cc:407] Connection header received
0812 07:12:37.045605 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:37.045611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:37.045684 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:37.045798 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:37.047132 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.047811 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.050000 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.050263 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.050375 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.052221 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:37.052225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:37.052228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:37.052246 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:37.052269 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:37.052278 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:37.052365 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:37.052643 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 07:12:37.052768 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:12:37.054807 23264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.042405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:39843 (local address 127.31.156.66:58167)
0812 07:12:37.043275 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:37.043340 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:37.045334 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:37.046012 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:37.046021 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:37.046127 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:37.046961 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.046980 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.047984 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.047992 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.049818 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.049838 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.050534 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.050541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.050880 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.051758 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:12:37.051763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:37.051843 (+ 80us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:12:37.051981 (+ 138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:37.052403 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:37.052419 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:37.052429 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 07:12:37.052597 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 07:12:37.053052 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":120,"threads_started":1}
I20260812 07:12:37.056712 23261 heartbeater.cc:347] Connected to a master server at 127.31.156.126:39843
I20260812 07:12:37.057355 23261 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:37.058182 23261 heartbeater.cc:511] Master 127.31.156.126:39843 requested a full tablet report, sending...
I20260812 07:12:37.059983 22888 ts_manager.cc:194] Registered new tserver with Master: c48a3f53acb74e568a2a45c4b9c4b001 (127.31.156.66:41919)
I20260812 07:12:37.060889 22888 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.66:58167
W20260812 07:12:37.181825 23267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:37.182240 23267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:37.182291 23267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:37.186630 23267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:37.186741 23267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:37.186803 23267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.156.67
I20260812 07:12:37.192310 23267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.156.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.156.67
--webserver_port=0
--tserver_master_addrs=127.31.156.126:39843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.156.67
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:37.193753 23267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:37.195513 23267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:37.208173 23274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:37.208187 23272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:37.208261 23267 server_base.cc:1034] running on GCE node
W20260812 07:12:37.208199 23277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:37.208917 23267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:37.209824 23267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:37.211131 23267 hybrid_clock.cc:648] HybridClock initialized: now 1786518757211039 us; error 94 us; skew 500 ppm
I20260812 07:12:37.213857 23267 webserver.cc:460] Webserver started at http://127.31.156.67:44179/ using document root <none> and password file <none>
I20260812 07:12:37.214545 23267 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:37.214619 23267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:37.214922 23267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:37.217567 23267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/data/instance:
uuid: "668310bf2f8044f9b58a1f768cf038c7"
format_stamp: "Formatted at 2026-08-12 07:12:37 on dist-test-slave-csg5"
server_key: "0e339ee5a9246ebb9b5d3e94ca367fbf"
server_key_iv: "30c500bf529269cef6750fd1e10e40a6"
server_key_version: "encryptionkey@0"
I20260812 07:12:37.218250 23267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance:
uuid: "668310bf2f8044f9b58a1f768cf038c7"
format_stamp: "Formatted at 2026-08-12 07:12:37 on dist-test-slave-csg5"
server_key: "0e339ee5a9246ebb9b5d3e94ca367fbf"
server_key_iv: "30c500bf529269cef6750fd1e10e40a6"
server_key_version: "encryptionkey@0"
I20260812 07:12:37.222848 23267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260812 07:12:37.226406 23283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:37.228375 23267 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260812 07:12:37.228528 23267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/wal
uuid: "668310bf2f8044f9b58a1f768cf038c7"
format_stamp: "Formatted at 2026-08-12 07:12:37 on dist-test-slave-csg5"
server_key: "0e339ee5a9246ebb9b5d3e94ca367fbf"
server_key_iv: "30c500bf529269cef6750fd1e10e40a6"
server_key_version: "encryptionkey@0"
I20260812 07:12:37.228781 23267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:37.247059 23267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:37.247918 23267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:37.248145 23267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:37.248847 23267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:37.249939 23267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:37.249990 23267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:37.250061 23267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:37.250094 23267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:37.261320 23267 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.67:45415
I20260812 07:12:37.261340 23409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.67:45415 every 8 connection(s)
I20260812 07:12:37.262462 23267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:37.264117 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 23267
I20260812 07:12:37.264232 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/wal/instance
I20260812 07:12:37.264710 32369 external_mini_cluster.cc:1428] Setting key 2419b4cf830e4491b17714bee01c5595
I20260812 07:12:37.275413 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.265670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:34959 (local address 127.31.156.126:39843)
0812 07:12:37.265870 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:37.265875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:37.267582 (+ 1707us) server_negotiation.cc:407] Connection header received
0812 07:12:37.269274 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:37.269278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:37.269346 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:37.269440 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:37.270730 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.271351 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.272799 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.273056 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.273197 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.274751 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:37.274756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:37.274759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:37.274787 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:37.274811 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:37.274821 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:37.274923 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:37.275144 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 07:12:37.275230 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:12:37.276851 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.266143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:39843 (local address 127.31.156.67:34959)
0812 07:12:37.267368 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:37.267464 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:37.268923 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:37.269671 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:37.269682 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:37.269799 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:37.270508 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.270526 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.271537 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.271540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.272586 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.272598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.273271 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.273276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.273480 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.274364 (+ 884us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:12:37.274367 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:37.274420 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:12:37.274505 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:37.274926 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:37.274941 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:37.274949 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:12:37.275181 (+ 232us) client_negotiation.cc:240] Negotiation successful
0812 07:12:37.275567 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":227,"threads_started":1}
I20260812 07:12:37.278390 23411 heartbeater.cc:347] Connected to a master server at 127.31.156.126:39843
I20260812 07:12:37.278781 23411 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:37.279543 23411 heartbeater.cc:511] Master 127.31.156.126:39843 requested a full tablet report, sending...
I20260812 07:12:37.280864 22888 ts_manager.cc:194] Registered new tserver with Master: 668310bf2f8044f9b58a1f768cf038c7 (127.31.156.67:45415)
I20260812 07:12:37.281490 22888 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.156.67:34959
I20260812 07:12:37.292120 32369 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:12:37.299288 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.294527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.31.156.126:39843)
0812 07:12:37.294720 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:37.294749 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:37.294874 (+ 125us) server_negotiation.cc:407] Connection header received
0812 07:12:37.295067 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:37.295069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:37.295116 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:37.295230 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:37.295593 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.296195 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.297251 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.297550 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.297641 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.298639 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:37.298642 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:37.298645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:37.298693 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:37.298766 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:37.298780 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:37.298905 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:37.299039 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 07:12:37.299114 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 07:12:37.303815 22888 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:12:37.307423 22888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:12:37.324573 23424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.317945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:41919 (local address 127.0.0.1:60534)
0812 07:12:37.318517 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:37.318577 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:37.318745 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:37.320212 (+ 1467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:37.320217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:37.320242 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:37.320323 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.320329 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.321746 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.321750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.322869 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.322880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.323988 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.323992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.324112 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.324157 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 07:12:37.324272 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 07:12:37.324406 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":96,"threads_started":1}
I20260812 07:12:37.324615 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.319223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.31.156.66:41919)
0812 07:12:37.319697 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:37.319711 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:37.319736 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:12:37.319816 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:37.319821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:37.320032 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:37.320225 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:37.320441 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.321607 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.323002 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.323845 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.323989 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.324213 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:37.324311 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 07:12:37.324421 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":151,"threads_started":1}
I20260812 07:12:37.324970 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.318821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55750 (local address 127.31.156.65:35099)
0812 07:12:37.319365 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:37.319373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:37.319403 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:12:37.319492 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:37.319499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:37.319719 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:37.320317 (+ 598us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:37.320490 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.321211 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.322904 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.323915 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.324009 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.324213 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:37.324673 (+ 460us) server_negotiation.cc:299] Negotiation successful
0812 07:12:37.324798 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":127,"threads_started":1}
I20260812 07:12:37.325481 23425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.318363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.67:45415 (local address 127.0.0.1:37608)
0812 07:12:37.318990 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:37.319000 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:37.319089 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:37.319909 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:37.319915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:37.319954 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:37.320187 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.320196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.322911 (+ 2715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.322917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.323726 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.323736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.324965 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.324970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.325088 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.325138 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 07:12:37.325224 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 07:12:37.325312 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":45,"spinlock_wait_cycles":5248,"thread_start_us":83,"threads_started":1}
I20260812 07:12:37.325558 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.318821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.31.156.67:45415)
0812 07:12:37.319476 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:37.319488 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:37.319506 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:12:37.319571 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:37.319575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:37.319718 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:37.320041 (+ 323us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:37.320330 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.321608 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.323868 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.324803 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.324944 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.325158 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:37.325256 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 07:12:37.325377 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":163,"threads_started":1}
I20260812 07:12:37.325901 23426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.318161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:35099 (local address 127.0.0.1:55750)
0812 07:12:37.318920 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:37.318934 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:37.319026 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:37.319909 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:37.319915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:37.319955 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:37.320168 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.320177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.321341 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.321347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.322726 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.322738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.324287 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.324290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.324385 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.324408 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:12:37.324493 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 07:12:37.324573 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":26,"spinlock_wait_cycles":8192,"thread_start_us":130,"threads_started":1}
I20260812 07:12:37.328649 23184 tablet_service.cc:1467] Processing CreateTablet for tablet 561558cac7c145afb573c0047645981d (DEFAULT_TABLE table=test-table [id=4bbdaf13488341488dca7f7264631fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:37.328649 23032 tablet_service.cc:1467] Processing CreateTablet for tablet 561558cac7c145afb573c0047645981d (DEFAULT_TABLE table=test-table [id=4bbdaf13488341488dca7f7264631fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:37.328648 23335 tablet_service.cc:1467] Processing CreateTablet for tablet 561558cac7c145afb573c0047645981d (DEFAULT_TABLE table=test-table [id=4bbdaf13488341488dca7f7264631fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:37.329807 23335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 561558cac7c145afb573c0047645981d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:37.329833 23032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 561558cac7c145afb573c0047645981d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:37.329825 23184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 561558cac7c145afb573c0047645981d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:37.337064 23430 tablet_bootstrap.cc:492] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7: Bootstrap starting.
I20260812 07:12:37.337855 23432 tablet_bootstrap.cc:492] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001: Bootstrap starting.
I20260812 07:12:37.339071 23431 tablet_bootstrap.cc:492] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e: Bootstrap starting.
I20260812 07:12:37.340162 23432 tablet_bootstrap.cc:654] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:37.340169 23430 tablet_bootstrap.cc:654] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:37.341099 23430 log.cc:824] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:37.341105 23432 log.cc:824] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:37.341521 23431 tablet_bootstrap.cc:654] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:37.342264 23431 log.cc:824] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:37.343583 23432 tablet_bootstrap.cc:492] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001: No bootstrap required, opened a new log
I20260812 07:12:37.343616 23430 tablet_bootstrap.cc:492] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7: No bootstrap required, opened a new log
I20260812 07:12:37.343820 23432 ts_tablet_manager.cc:1397] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260812 07:12:37.343822 23430 ts_tablet_manager.cc:1397] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:12:37.344336 23431 tablet_bootstrap.cc:492] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e: No bootstrap required, opened a new log
I20260812 07:12:37.344538 23431 ts_tablet_manager.cc:1397] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 07:12:37.348464 23432 raft_consensus.cc:348] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41919 } } peers { permanent_uuid: "9e5bfde8100e4636a759cff05140891e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35099 } } peers { permanent_uuid: "668310bf2f8044f9b58a1f768cf038c7" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45415 } }
I20260812 07:12:37.348817 23432 raft_consensus.cc:374] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:37.348606 23430 raft_consensus.cc:348] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41919 } } peers { permanent_uuid: "9e5bfde8100e4636a759cff05140891e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35099 } } peers { permanent_uuid: "668310bf2f8044f9b58a1f768cf038c7" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45415 } }
I20260812 07:12:37.348879 23432 raft_consensus.cc:729] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48a3f53acb74e568a2a45c4b9c4b001, State: Initialized, Role: FOLLOWER
I20260812 07:12:37.348901 23430 raft_consensus.cc:374] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:37.348950 23430 raft_consensus.cc:729] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668310bf2f8044f9b58a1f768cf038c7, State: Initialized, Role: FOLLOWER
I20260812 07:12:37.348886 23431 raft_consensus.cc:348] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41919 } } peers { permanent_uuid: "9e5bfde8100e4636a759cff05140891e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35099 } } peers { permanent_uuid: "668310bf2f8044f9b58a1f768cf038c7" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45415 } }
I20260812 07:12:37.349165 23431 raft_consensus.cc:374] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:37.349223 23431 raft_consensus.cc:729] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5bfde8100e4636a759cff05140891e, State: Initialized, Role: FOLLOWER
I20260812 07:12:37.349478 23430 consensus_queue.cc:260] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41919 } } peers { permanent_uuid: "9e5bfde8100e4636a759cff05140891e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35099 } } peers { permanent_uuid: "668310bf2f8044f9b58a1f768cf038c7" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45415 } }
I20260812 07:12:37.349478 23432 consensus_queue.cc:260] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41919 } } peers { permanent_uuid: "9e5bfde8100e4636a759cff05140891e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35099 } } peers { permanent_uuid: "668310bf2f8044f9b58a1f768cf038c7" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45415 } }
I20260812 07:12:37.349603 23431 consensus_queue.cc:260] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41919 } } peers { permanent_uuid: "9e5bfde8100e4636a759cff05140891e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35099 } } peers { permanent_uuid: "668310bf2f8044f9b58a1f768cf038c7" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45415 } }
I20260812 07:12:37.350580 23411 heartbeater.cc:503] Master 127.31.156.126:39843 was elected leader, sending a full tablet report...
I20260812 07:12:37.350776 23430 ts_tablet_manager.cc:1428] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 07:12:37.350776 23431 ts_tablet_manager.cc:1428] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260812 07:12:37.350893 23106 heartbeater.cc:503] Master 127.31.156.126:39843 was elected leader, sending a full tablet report...
I20260812 07:12:37.351989 23261 heartbeater.cc:503] Master 127.31.156.126:39843 was elected leader, sending a full tablet report...
I20260812 07:12:37.353755 23432 ts_tablet_manager.cc:1428] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260812 07:12:37.413836 23436 raft_consensus.cc:482] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:37.414072 23436 raft_consensus.cc:504] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41919 } } peers { permanent_uuid: "9e5bfde8100e4636a759cff05140891e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35099 } } peers { permanent_uuid: "668310bf2f8044f9b58a1f768cf038c7" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45415 } }
I20260812 07:12:37.415493 23436 leader_election.cc:288] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c48a3f53acb74e568a2a45c4b9c4b001 (127.31.156.66:41919), 9e5bfde8100e4636a759cff05140891e (127.31.156.65:35099)
I20260812 07:12:37.420352 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.415910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:40355 (local address 127.31.156.66:41919)
0812 07:12:37.416041 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:37.416045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:37.416062 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:12:37.416144 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:37.416148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:37.416206 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:37.416318 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:37.416672 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.417539 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.419073 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.419654 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.419802 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.419859 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:37.420107 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 07:12:37.420178 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:12:37.420446 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.415719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.66:41919 (local address 127.31.156.67:40355)
0812 07:12:37.415877 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:37.415895 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:37.416006 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:37.416349 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:37.416355 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:37.416397 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:37.416539 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.416549 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.417635 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.417638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.418935 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.418947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.419816 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.419819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.419907 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.419972 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 07:12:37.420072 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 07:12:37.420270 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":1}
I20260812 07:12:37.420543 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.416061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.156.67:44715 (local address 127.31.156.65:35099)
0812 07:12:37.416181 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:37.416185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:37.416408 (+ 223us) server_negotiation.cc:407] Connection header received
0812 07:12:37.416581 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:37.416583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:37.416658 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:37.416741 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:37.417153 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.417815 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.419246 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.419857 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.419969 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.420004 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:37.420243 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 07:12:37.420425 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:12:37.420971 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.415864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:35099 (local address 127.31.156.67:44715)
0812 07:12:37.416303 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:37.416320 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:37.416466 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:37.416821 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:37.416832 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:37.416853 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:37.416978 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.416983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.417926 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.417929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.419137 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:37.419145 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.420000 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.420006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:37.420119 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.420140 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 07:12:37.420214 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 07:12:37.420277 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":12,"thread_start_us":170,"threads_started":1}
I20260812 07:12:37.421366 23208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "561558cac7c145afb573c0047645981d" candidate_uuid: "668310bf2f8044f9b58a1f768cf038c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" is_pre_election: true
I20260812 07:12:37.421720 23053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "561558cac7c145afb573c0047645981d" candidate_uuid: "668310bf2f8044f9b58a1f768cf038c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5bfde8100e4636a759cff05140891e" is_pre_election: true
I20260812 07:12:37.421828 23208 raft_consensus.cc:2454] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 668310bf2f8044f9b58a1f768cf038c7 in term 0.
I20260812 07:12:37.422024 23053 raft_consensus.cc:2454] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 668310bf2f8044f9b58a1f768cf038c7 in term 0.
I20260812 07:12:37.422722 23285 leader_election.cc:302] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668310bf2f8044f9b58a1f768cf038c7, 9e5bfde8100e4636a759cff05140891e; no voters:
I20260812 07:12:37.423372 23436 raft_consensus.cc:2781] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:12:37.423455 23436 raft_consensus.cc:482] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:12:37.423486 23436 raft_consensus.cc:3037] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:37.424696 23436 raft_consensus.cc:504] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41919 } } peers { permanent_uuid: "9e5bfde8100e4636a759cff05140891e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35099 } } peers { permanent_uuid: "668310bf2f8044f9b58a1f768cf038c7" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45415 } }
I20260812 07:12:37.425510 23436 leader_election.cc:288] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [CANDIDATE]: Term 1 election: Requested vote from peers c48a3f53acb74e568a2a45c4b9c4b001 (127.31.156.66:41919), 9e5bfde8100e4636a759cff05140891e (127.31.156.65:35099)
I20260812 07:12:37.426028 23208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "561558cac7c145afb573c0047645981d" candidate_uuid: "668310bf2f8044f9b58a1f768cf038c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48a3f53acb74e568a2a45c4b9c4b001"
I20260812 07:12:37.426214 23208 raft_consensus.cc:3037] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:37.426204 23053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "561558cac7c145afb573c0047645981d" candidate_uuid: "668310bf2f8044f9b58a1f768cf038c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5bfde8100e4636a759cff05140891e"
I20260812 07:12:37.426581 23053 raft_consensus.cc:3037] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:37.427858 23208 raft_consensus.cc:2454] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668310bf2f8044f9b58a1f768cf038c7 in term 1.
I20260812 07:12:37.428078 23053 raft_consensus.cc:2454] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668310bf2f8044f9b58a1f768cf038c7 in term 1.
I20260812 07:12:37.428385 23287 leader_election.cc:302] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668310bf2f8044f9b58a1f768cf038c7, c48a3f53acb74e568a2a45c4b9c4b001; no voters:
I20260812 07:12:37.428620 23436 raft_consensus.cc:2781] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:37.428934 23436 raft_consensus.cc:686] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 1 LEADER]: Becoming Leader. State: Replica: 668310bf2f8044f9b58a1f768cf038c7, State: Running, Role: LEADER
I20260812 07:12:37.429378 23436 consensus_queue.cc:237] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41919 } } peers { permanent_uuid: "9e5bfde8100e4636a759cff05140891e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35099 } } peers { permanent_uuid: "668310bf2f8044f9b58a1f768cf038c7" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45415 } }
I20260812 07:12:37.433121 22888 catalog_manager.cc:5495] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 668310bf2f8044f9b58a1f768cf038c7 (127.31.156.67). New cstate: current_term: 1 leader_uuid: "668310bf2f8044f9b58a1f768cf038c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e5bfde8100e4636a759cff05140891e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "668310bf2f8044f9b58a1f768cf038c7" member_type: VOTER last_known_addr { host: "127.31.156.67" port: 45415 } health_report { overall_health: HEALTHY } } }
I20260812 07:12:37.458017 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.454165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51576 (local address 127.31.156.67:45415)
0812 07:12:37.454336 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:37.454341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:37.454357 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:12:37.454472 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:37.454474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:37.454534 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:37.454657 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:12:37.454954 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.455482 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.456387 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:37.456611 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:37.456726 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:37.457076 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:12:37.457547 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:12:37.457692 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:37.457757 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 07:12:37.457831 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:12:37.465663 23208 raft_consensus.cc:1261] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001 [term 1 FOLLOWER]: Refusing update from remote peer 668310bf2f8044f9b58a1f768cf038c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:37.465663 23053 raft_consensus.cc:1261] T 561558cac7c145afb573c0047645981d P 9e5bfde8100e4636a759cff05140891e [term 1 FOLLOWER]: Refusing update from remote peer 668310bf2f8044f9b58a1f768cf038c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:37.466863 23436 consensus_queue.cc:1035] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5bfde8100e4636a759cff05140891e" member_type: VOTER last_known_addr { host: "127.31.156.65" port: 35099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:37.467203 23440 consensus_queue.cc:1035] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48a3f53acb74e568a2a45c4b9c4b001" member_type: VOTER last_known_addr { host: "127.31.156.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:37.481262 23445 mvcc.cc:205] Tried to move back new op lower bound from 7317580830568554496 to 7317580830433853440. Current Snapshot: MvccSnapshot[applied={T|T < 7317580830568554496 or (T in {7317580830568554496})}]
I20260812 07:12:37.488914 22890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:37.489159 22890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:37.492650 22890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86aaa9ff64734c7683e669f0a2815e0e: Sending DeleteTablet for 3 replicas of tablet 561558cac7c145afb573c0047645981d
I20260812 07:12:37.493854 23184 tablet_service.cc:1514] Processing DeleteTablet for tablet 561558cac7c145afb573c0047645981d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:37 UTC) from {username='slave'} at 127.0.0.1:60534
I20260812 07:12:37.494081 23335 tablet_service.cc:1514] Processing DeleteTablet for tablet 561558cac7c145afb573c0047645981d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:37 UTC) from {username='slave'} at 127.0.0.1:37608
I20260812 07:12:37.494516 23032 tablet_service.cc:1514] Processing DeleteTablet for tablet 561558cac7c145afb573c0047645981d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:37 UTC) from {username='slave'} at 127.0.0.1:55750
I20260812 07:12:37.494656 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 22960
I20260812 07:12:37.500073 23456 tablet_replica.cc:330] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001: stopping tablet replica
I20260812 07:12:37.500500 23456 raft_consensus.cc:2229] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:37.500921 23456 raft_consensus.cc:2258] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:37.501540 23457 tablet_replica.cc:330] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7: stopping tablet replica
I20260812 07:12:37.501901 23457 raft_consensus.cc:2229] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:12:37.502326 23457 raft_consensus.cc:2258] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:37.503319 23456 ts_tablet_manager.cc:1905] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:37.503927 23457 ts_tablet_manager.cc:1905] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:37.507229 23457 ts_tablet_manager.cc:1918] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:37.507370 23457 log.cc:1196] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-2/wal/wals/561558cac7c145afb573c0047645981d
I20260812 07:12:37.507550 23456 ts_tablet_manager.cc:1918] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:37.507730 23457 ts_tablet_manager.cc:1939] T 561558cac7c145afb573c0047645981d P 668310bf2f8044f9b58a1f768cf038c7: Deleting consensus metadata
I20260812 07:12:37.507740 23456 log.cc:1196] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001: Deleting WAL directory at /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518468557119-32369-0/minicluster-data/ts-1/wal/wals/561558cac7c145afb573c0047645981d
I20260812 07:12:37.508189 23456 ts_tablet_manager.cc:1939] T 561558cac7c145afb573c0047645981d P c48a3f53acb74e568a2a45c4b9c4b001: Deleting consensus metadata
I20260812 07:12:37.509459 22873 catalog_manager.cc:4841] TS 668310bf2f8044f9b58a1f768cf038c7 (127.31.156.67:45415): tablet 561558cac7c145afb573c0047645981d (table test-table [id=4bbdaf13488341488dca7f7264631fe8]) successfully deleted
I20260812 07:12:37.509649 22874 catalog_manager.cc:4841] TS c48a3f53acb74e568a2a45c4b9c4b001 (127.31.156.66:41919): tablet 561558cac7c145afb573c0047645981d (table test-table [id=4bbdaf13488341488dca7f7264631fe8]) successfully deleted
W20260812 07:12:37.510098 22871 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.156.65:35099 (error 108)
I20260812 07:12:37.510401 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 23110
I20260812 07:12:37.514631 23426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:37.514123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.65:35099 (local address 127.0.0.1:59852)
0812 07:12:37.514340 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:37.514482 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.65:35099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 07:12:37.515129 22871 catalog_manager.cc:4584] TS 9e5bfde8100e4636a759cff05140891e (127.31.156.65:35099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 561558cac7c145afb573c0047645981d: Network error: Client connection negotiation failed: client connection to 127.31.156.65:35099: connect: Connection refused (error 111)
I20260812 07:12:37.526470 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 23267
I20260812 07:12:37.536602 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 22852
2026-08-12T07:12:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1238 ms)
[----------] 6 tests from EncryptionPolicyTest (12392 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:37 dist-test-slave-csg5 krb5kdc[23463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:37 dist-test-slave-csg5 krb5kdc[23463](info): set up 2 sockets
Aug 12 07:12:37 dist-test-slave-csg5 krb5kdc[23463](info): commencing operation
krb5kdc: starting...
W20260812 07:12:38.714422 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:38 dist-test-slave-csg5 krb5kdc[23463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:38.871495 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:38.997175 23477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:38.997555 23477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:38.997635 23477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:39.002061 23477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:39.002197 23477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:39.002234 23477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:39.002290 23477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:39.002345 23477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:39.008293 23477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37503
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:39.009645 23477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:39.010785 23477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:39.020684 23485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:39.020695 23483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:39.020684 23482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:39.021014 23477 server_base.cc:1034] running on GCE node
I20260812 07:12:39.021791 23477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:39.023142 23477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:39.024377 23477 hybrid_clock.cc:648] HybridClock initialized: now 1786518759024356 us; error 38 us; skew 500 ppm
Aug 12 07:12:39 dist-test-slave-csg5 krb5kdc[23463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518759, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:39.027807 23477 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:39.028805 23477 webserver.cc:460] Webserver started at http://127.0.0.1:44037/ using document root <none> and password file <none>
I20260812 07:12:39.029399 23477 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:39.029474 23477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:39.029678 23477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:39.032114 23477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "50f9becf69d2482483a72e9ecd9b35e0"
format_stamp: "Formatted at 2026-08-12 07:12:39 on dist-test-slave-csg5"
server_key: "de27159329429a9e1364a393a26caa6f"
server_key_iv: "091833f60319fefff46bbebcc541caa1"
server_key_version: "encryptionkey@0"
I20260812 07:12:39.032730 23477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "50f9becf69d2482483a72e9ecd9b35e0"
format_stamp: "Formatted at 2026-08-12 07:12:39 on dist-test-slave-csg5"
server_key: "de27159329429a9e1364a393a26caa6f"
server_key_iv: "091833f60319fefff46bbebcc541caa1"
server_key_version: "encryptionkey@0"
I20260812 07:12:39.037118 23477 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:12:39.039995 23492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:39.041581 23477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:39.041718 23477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "50f9becf69d2482483a72e9ecd9b35e0"
format_stamp: "Formatted at 2026-08-12 07:12:39 on dist-test-slave-csg5"
server_key: "de27159329429a9e1364a393a26caa6f"
server_key_iv: "091833f60319fefff46bbebcc541caa1"
server_key_version: "encryptionkey@0"
I20260812 07:12:39.041883 23477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:39.061147 23477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:39.065186 23477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:39.065521 23477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:39.073839 23477 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37503
I20260812 07:12:39.073869 23552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37503 every 8 connection(s)
I20260812 07:12:39.075074 23477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:39.078114 23553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:39.081156 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 23477
I20260812 07:12:39.081295 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:39.081616 32369 external_mini_cluster.cc:1428] Setting key f40d3fb90368b0b4394e89b988468045
I20260812 07:12:39.085090 23553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0: Bootstrap starting.
I20260812 07:12:39.088088 23553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:39.089126 23553 log.cc:824] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:39.092460 23553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0: No bootstrap required, opened a new log
Aug 12 07:12:39 dist-test-slave-csg5 krb5kdc[23463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:12:39.096346 23553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f9becf69d2482483a72e9ecd9b35e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260812 07:12:39.096647 23553 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:39.096721 23553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f9becf69d2482483a72e9ecd9b35e0, State: Initialized, Role: FOLLOWER
I20260812 07:12:39.097316 23553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f9becf69d2482483a72e9ecd9b35e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260812 07:12:39.097502 23553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:39.097601 23553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:39.097748 23553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:39.098958 23553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f9becf69d2482483a72e9ecd9b35e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260812 07:12:39.099439 23553 leader_election.cc:302] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50f9becf69d2482483a72e9ecd9b35e0; no voters:
I20260812 07:12:39.099954 23553 leader_election.cc:288] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:39.100104 23558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:39.100333 23558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [term 1 LEADER]: Becoming Leader. State: Replica: 50f9becf69d2482483a72e9ecd9b35e0, State: Running, Role: LEADER
I20260812 07:12:39.100497 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:39.083310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56069 (local address 127.0.0.1:37503)
0812 07:12:39.083925 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:39.083950 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:39.084001 (+ 51us) server_negotiation.cc:407] Connection header received
0812 07:12:39.084874 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:39.084898 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:39.085379 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:39.085776 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:39.086571 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:39.088171 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:39.089497 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:39.090270 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:39.090410 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:39.094029 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:39.094036 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:39.094052 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:39.094087 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:39.096827 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:39.097467 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:39.097478 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:39.097504 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:39.097668 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:39.098078 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:39.098081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:39.098083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:39.098540 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:39.098830 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:39.099190 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 07:12:39.099440 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":173,"threads_started":1}
I20260812 07:12:39.100684 23558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f9becf69d2482483a72e9ecd9b35e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260812 07:12:39.101500 23553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:39.102070 23558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50f9becf69d2482483a72e9ecd9b35e0. Latest consensus state: current_term: 1 leader_uuid: "50f9becf69d2482483a72e9ecd9b35e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f9becf69d2482483a72e9ecd9b35e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } } }
I20260812 07:12:39.102208 23558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:39.102726 23559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50f9becf69d2482483a72e9ecd9b35e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f9becf69d2482483a72e9ecd9b35e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } } }
I20260812 07:12:39.102893 23559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:39.104300 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 07:12:39.107890 23575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:12:39.108008 23575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:12:39.108084 23563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:39.109133 23563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:39.116684 23563 catalog_manager.cc:1347] Generated new cluster ID: 1bb4e8fa74e44c9eaba0ed626c137ecb
I20260812 07:12:39.116777 23563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:39.117815 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:39.107018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57805 (local address 127.0.0.1:37503)
0812 07:12:39.107153 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:39.107170 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:39.107420 (+ 250us) server_negotiation.cc:407] Connection header received
0812 07:12:39.107580 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:39.107583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:39.107638 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:39.107739 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:39.108667 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:39.109309 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:39.110231 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:39.110504 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:39.110630 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:39.112599 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:39.112605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:39.112611 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:39.112650 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:39.114805 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:39.115749 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:39.115754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:39.115756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:39.115884 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:39.116581 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:39.116588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:39.116591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:39.116873 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:39.117137 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:39.117499 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 07:12:39.117626 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:12:39.136706 23563 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:39.137826 23563 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:39.144124 23563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50f9becf69d2482483a72e9ecd9b35e0: Generated new TSK 0
I20260812 07:12:39.145012 23563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:39.159935 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 23477
W20260812 07:12:39.169001 23460 connection.cc:535] client connection to 127.0.0.1:37503 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37503): Connection reset by peer (error 104)
2026-08-12T07:12:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1621 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:39 dist-test-slave-csg5 krb5kdc[23587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:39 dist-test-slave-csg5 krb5kdc[23587](info): set up 2 sockets
Aug 12 07:12:39 dist-test-slave-csg5 krb5kdc[23587](info): commencing operation
krb5kdc: starting...
W20260812 07:12:40.375871 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:40 dist-test-slave-csg5 krb5kdc[23587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:40.530937 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42007
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:40.655826 23604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:40.656209 23604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:40.656320 23604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:40.660831 23604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:40.660948 23604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:40.660987 23604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:40.661043 23604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:40.661082 23604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:40.667045 23604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42007
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:40.668356 23604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:40.669661 23604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:40.679714 23614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:40.679788 23616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:40.679860 23604 server_base.cc:1034] running on GCE node
W20260812 07:12:40.679788 23613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:40.680698 23604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:40.682034 23604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:40.683419 23604 hybrid_clock.cc:648] HybridClock initialized: now 1786518760683381 us; error 56 us; skew 500 ppm
Aug 12 07:12:40 dist-test-slave-csg5 krb5kdc[23587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518760, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:40.697098 23604 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:40.698556 23604 webserver.cc:460] Webserver started at http://127.0.0.1:34079/ using document root <none> and password file <none>
I20260812 07:12:40.699321 23604 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:40.699438 23604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:40.699744 23604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:40.702343 23604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "725a4506596a45868a48c89b536017a8"
format_stamp: "Formatted at 2026-08-12 07:12:40 on dist-test-slave-csg5"
server_key: "26c0e827afa750e464ff17e5dc146aee"
server_key_iv: "d6f033a59f7b70668f32a90894c1c674"
server_key_version: "encryptionkey@0"
I20260812 07:12:40.703118 23604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "725a4506596a45868a48c89b536017a8"
format_stamp: "Formatted at 2026-08-12 07:12:40 on dist-test-slave-csg5"
server_key: "26c0e827afa750e464ff17e5dc146aee"
server_key_iv: "d6f033a59f7b70668f32a90894c1c674"
server_key_version: "encryptionkey@0"
I20260812 07:12:40.708204 23604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:12:40.711642 23623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:40.713491 23604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260812 07:12:40.713730 23604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "725a4506596a45868a48c89b536017a8"
format_stamp: "Formatted at 2026-08-12 07:12:40 on dist-test-slave-csg5"
server_key: "26c0e827afa750e464ff17e5dc146aee"
server_key_iv: "d6f033a59f7b70668f32a90894c1c674"
server_key_version: "encryptionkey@0"
I20260812 07:12:40.713927 23604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:40.761647 23604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:40.766119 23604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:40.766417 23604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:40.775794 23604 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42007
I20260812 07:12:40.775826 23681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42007 every 8 connection(s)
I20260812 07:12:40.776953 23604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:40.782308 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 23604
I20260812 07:12:40.782302 23682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:40.782554 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:40.782982 32369 external_mini_cluster.cc:1428] Setting key 0ceac20d858d7ace4ed53dcff63e40c4
I20260812 07:12:40.790444 23682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8: Bootstrap starting.
I20260812 07:12:40.793004 23682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:40.793996 23682 log.cc:824] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:40.794792 23686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.784652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38825 (local address 127.0.0.1:42007)
0812 07:12:40.785255 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.785272 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.785359 (+ 87us) server_negotiation.cc:407] Connection header received
0812 07:12:40.786874 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.786901 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.787389 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.787867 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.788335 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.789430 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.790692 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.791331 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.791548 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.792579 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.792585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.792600 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:40.793022 (+ 422us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.793087 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:40.793112 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.793288 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.793519 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.793700 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":196,"threads_started":1}
I20260812 07:12:40.796584 23682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8: No bootstrap required, opened a new log
I20260812 07:12:40.798419 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:40.801239 23682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725a4506596a45868a48c89b536017a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260812 07:12:40.801585 23682 raft_consensus.cc:374] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:40.801678 23682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 725a4506596a45868a48c89b536017a8, State: Initialized, Role: FOLLOWER
I20260812 07:12:40.802212 23682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725a4506596a45868a48c89b536017a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260812 07:12:40.802402 23682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:40.802476 23682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:40.802603 23682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:40.803982 23682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725a4506596a45868a48c89b536017a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260812 07:12:40.804440 23682 leader_election.cc:302] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 725a4506596a45868a48c89b536017a8; no voters:
I20260812 07:12:40.804836 23682 leader_election.cc:288] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:40.804927 23694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:40.805150 23694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [term 1 LEADER]: Becoming Leader. State: Replica: 725a4506596a45868a48c89b536017a8, State: Running, Role: LEADER
I20260812 07:12:40.805490 23694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725a4506596a45868a48c89b536017a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260812 07:12:40.805928 23686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.800833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40463 (local address 127.0.0.1:42007)
0812 07:12:40.800977 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.800982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.801005 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:12:40.801142 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.801145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.801196 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.801295 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.801662 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.802406 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.803619 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.803872 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.803962 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.805171 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.805175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.805181 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:40.805207 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.805317 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:40.805334 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.805522 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.805722 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.805770 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:12:40.806380 23682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:40.807301 23695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "725a4506596a45868a48c89b536017a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725a4506596a45868a48c89b536017a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } } }
I20260812 07:12:40.807461 23695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:40.807696 23695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 725a4506596a45868a48c89b536017a8. Latest consensus state: current_term: 1 leader_uuid: "725a4506596a45868a48c89b536017a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725a4506596a45868a48c89b536017a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } } }
I20260812 07:12:40.807786 23695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:40.809245 23703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:40.812342 23703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:40.819101 23703 catalog_manager.cc:1347] Generated new cluster ID: 03bbae862b9444eaadcd632e02624f8d
I20260812 07:12:40.819221 23703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:40.829303 23703 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:40.830845 23703 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:40.837641 23703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 725a4506596a45868a48c89b536017a8: Generated new TSK 0
I20260812 07:12:40.838606 23703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:40.849292 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 23604
W20260812 07:12:40.861604 23585 connection.cc:535] client connection to 127.0.0.1:42007 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42007): Connection reset by peer (error 104)
2026-08-12T07:12:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1692 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:40 dist-test-slave-csg5 krb5kdc[23715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:40 dist-test-slave-csg5 krb5kdc[23715](info): set up 2 sockets
Aug 12 07:12:40 dist-test-slave-csg5 krb5kdc[23715](info): commencing operation
krb5kdc: starting...
W20260812 07:12:42.048599 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:42 dist-test-slave-csg5 krb5kdc[23715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:42.207883 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46765
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:42.335067 23731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:42.335467 23731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:42.335556 23731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:42.339855 23731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:42.339984 23731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:42.340022 23731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:42.340061 23731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:42.340093 23731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:42.345898 23731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46765
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:42.347226 23731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:42.348506 23731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:42.360512 23737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:42.360658 23739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:42.360662 23736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:42.360782 23731 server_base.cc:1034] running on GCE node
I20260812 07:12:42.361627 23731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:42.363266 23731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:42.364476 23731 hybrid_clock.cc:648] HybridClock initialized: now 1786518762364440 us; error 52 us; skew 500 ppm
Aug 12 07:12:42 dist-test-slave-csg5 krb5kdc[23715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518762, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:42.368083 23731 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:42.369282 23731 webserver.cc:460] Webserver started at http://127.0.0.1:42323/ using document root <none> and password file <none>
I20260812 07:12:42.369925 23731 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:42.370026 23731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:42.370289 23731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:42.372712 23731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "d277bc21cebc441bb5209672b9aed701"
format_stamp: "Formatted at 2026-08-12 07:12:42 on dist-test-slave-csg5"
server_key: "82c962ad9c21bd6a84aebf5dcf78a3cb"
server_key_iv: "2ae7ff0e612d735c746652771e2615ed"
server_key_version: "encryptionkey@0"
I20260812 07:12:42.373358 23731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "d277bc21cebc441bb5209672b9aed701"
format_stamp: "Formatted at 2026-08-12 07:12:42 on dist-test-slave-csg5"
server_key: "82c962ad9c21bd6a84aebf5dcf78a3cb"
server_key_iv: "2ae7ff0e612d735c746652771e2615ed"
server_key_version: "encryptionkey@0"
I20260812 07:12:42.377856 23731 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:12:42.380744 23746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:42.382268 23731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:12:42.382448 23731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "d277bc21cebc441bb5209672b9aed701"
format_stamp: "Formatted at 2026-08-12 07:12:42 on dist-test-slave-csg5"
server_key: "82c962ad9c21bd6a84aebf5dcf78a3cb"
server_key_iv: "2ae7ff0e612d735c746652771e2615ed"
server_key_version: "encryptionkey@0"
I20260812 07:12:42.382602 23731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:42.388015 23731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:42.391598 23731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:42.391884 23731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:42.399955 23731 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46765
I20260812 07:12:42.400038 23806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46765 every 8 connection(s)
I20260812 07:12:42.401041 23731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:42.404389 23807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:42.407627 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 23731
I20260812 07:12:42.407814 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:42.408126 32369 external_mini_cluster.cc:1428] Setting key a8e34887b60b9740ae849577e55289e1
I20260812 07:12:42.411361 23807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701: Bootstrap starting.
I20260812 07:12:42.414286 23807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:42.414489 23810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:42.409646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51247 (local address 127.0.0.1:46765)
0812 07:12:42.410169 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:42.410192 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:42.410239 (+ 47us) server_negotiation.cc:407] Connection header received
0812 07:12:42.411137 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:42.411162 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:42.411557 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:42.411969 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:42.413006 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:42.413011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:42.413024 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:42.413036 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:42.413097 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:42.413117 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:42.413245 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:42.413459 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 07:12:42.413648 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":154,"threads_started":1}
I20260812 07:12:42.415213 23807 log.cc:824] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:42.417723 23807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701: No bootstrap required, opened a new log
I20260812 07:12:42.418082 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:42.422447 23807 raft_consensus.cc:348] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d277bc21cebc441bb5209672b9aed701" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46765 } }
I20260812 07:12:42.422864 23807 raft_consensus.cc:374] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:42.422977 23807 raft_consensus.cc:729] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d277bc21cebc441bb5209672b9aed701, State: Initialized, Role: FOLLOWER
I20260812 07:12:42.423537 23807 consensus_queue.cc:260] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d277bc21cebc441bb5209672b9aed701" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46765 } }
I20260812 07:12:42.423676 23807 raft_consensus.cc:388] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:42.423765 23807 raft_consensus.cc:482] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:42.423894 23807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:42.424383 23810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:42.422172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39583 (local address 127.0.0.1:46765)
0812 07:12:42.422325 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:42.422330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:42.422564 (+ 234us) server_negotiation.cc:407] Connection header received
0812 07:12:42.422744 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:42.422747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:42.422794 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:42.422892 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:42.423965 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:42.423970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:42.423972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:42.423989 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:42.424021 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:42.424035 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:42.424141 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:42.424202 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 07:12:42.424248 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:12:42.425283 23807 raft_consensus.cc:504] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d277bc21cebc441bb5209672b9aed701" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46765 } }
I20260812 07:12:42.425662 23807 leader_election.cc:302] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d277bc21cebc441bb5209672b9aed701; no voters:
I20260812 07:12:42.425997 23807 leader_election.cc:288] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:42.426154 23817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:42.426435 23817 raft_consensus.cc:686] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [term 1 LEADER]: Becoming Leader. State: Replica: d277bc21cebc441bb5209672b9aed701, State: Running, Role: LEADER
I20260812 07:12:42.426870 23817 consensus_queue.cc:237] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d277bc21cebc441bb5209672b9aed701" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46765 } }
I20260812 07:12:42.427410 23807 sys_catalog.cc:564] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:42.428716 23818 sys_catalog.cc:455] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d277bc21cebc441bb5209672b9aed701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d277bc21cebc441bb5209672b9aed701" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46765 } } }
I20260812 07:12:42.429385 23819 sys_catalog.cc:455] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d277bc21cebc441bb5209672b9aed701. Latest consensus state: current_term: 1 leader_uuid: "d277bc21cebc441bb5209672b9aed701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d277bc21cebc441bb5209672b9aed701" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46765 } } }
I20260812 07:12:42.429505 23819 sys_catalog.cc:458] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:42.429968 23818 sys_catalog.cc:458] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701 [sys.catalog]: This master's current role is: LEADER
W20260812 07:12:42.432572 23832 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:12:42.432663 23832 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:12:42.432754 23833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:42.433539 23833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:42.440421 23833 catalog_manager.cc:1347] Generated new cluster ID: 426b44e8f3e64a0fb290a91b4fb31cf2
I20260812 07:12:42.440524 23833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:42.472618 23833 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:42.473011 23833 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:42.487275 23833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d277bc21cebc441bb5209672b9aed701: Generated new TSK 0
I20260812 07:12:42.488114 23833 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:42.496745 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 23731
2026-08-12T07:12:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1643 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:42 dist-test-slave-csg5 krb5kdc[23840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:42 dist-test-slave-csg5 krb5kdc[23840](info): set up 2 sockets
Aug 12 07:12:42 dist-test-slave-csg5 krb5kdc[23840](info): commencing operation
krb5kdc: starting...
W20260812 07:12:43.689321 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:43 dist-test-slave-csg5 krb5kdc[23840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:43.839380 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:43.965152 23854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:43.965615 23854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:43.965710 23854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:43.969965 23854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:43.970093 23854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:43.970132 23854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:43.970171 23854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:43.970206 23854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:43.975883 23854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:43.977137 23854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:43.978302 23854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:43.988097 23861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:43.988101 23860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:43.988333 23854 server_base.cc:1034] running on GCE node
W20260812 07:12:43.988128 23863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:43.989250 23854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:43.990641 23854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:43.991945 23854 hybrid_clock.cc:648] HybridClock initialized: now 1786518763991900 us; error 62 us; skew 500 ppm
Aug 12 07:12:43 dist-test-slave-csg5 krb5kdc[23840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518763, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:43.995687 23854 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:43.996841 23854 webserver.cc:460] Webserver started at http://127.0.0.1:40097/ using document root <none> and password file <none>
I20260812 07:12:43.997478 23854 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:43.997586 23854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:43.997874 23854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:44.000383 23854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "62d57d02677e428194452a2510737749"
format_stamp: "Formatted at 2026-08-12 07:12:43 on dist-test-slave-csg5"
server_key: "6e4912f9f19eb5946f635c2bb3631d93"
server_key_iv: "bfae6c3c146cb64cc3755df6ab84602e"
server_key_version: "encryptionkey@0"
I20260812 07:12:44.001111 23854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "62d57d02677e428194452a2510737749"
format_stamp: "Formatted at 2026-08-12 07:12:43 on dist-test-slave-csg5"
server_key: "6e4912f9f19eb5946f635c2bb3631d93"
server_key_iv: "bfae6c3c146cb64cc3755df6ab84602e"
server_key_version: "encryptionkey@0"
I20260812 07:12:44.006031 23854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 07:12:44.009112 23870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:44.010828 23854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:12:44.011031 23854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "62d57d02677e428194452a2510737749"
format_stamp: "Formatted at 2026-08-12 07:12:43 on dist-test-slave-csg5"
server_key: "6e4912f9f19eb5946f635c2bb3631d93"
server_key_iv: "bfae6c3c146cb64cc3755df6ab84602e"
server_key_version: "encryptionkey@0"
I20260812 07:12:44.011190 23854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:44.032709 23854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:44.036732 23854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:44.037003 23854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:44.046342 23854 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45977
I20260812 07:12:44.046469 23929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45977 every 8 connection(s)
I20260812 07:12:44.047622 23854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:44.049034 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 23854
I20260812 07:12:44.049193 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:44.049561 32369 external_mini_cluster.cc:1428] Setting key 446338d3dbb49fbe45497601994937b9
I20260812 07:12:44.051666 23931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:44.057940 23931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749: Bootstrap starting.
Aug 12 07:12:44 dist-test-slave-csg5 krb5kdc[23840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:12:44.060603 23931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:44.061702 23931 log.cc:824] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:44.064759 23931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749: No bootstrap required, opened a new log
I20260812 07:12:44.066423 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:44.051210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33975 (local address 127.0.0.1:45977)
0812 07:12:44.051795 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:44.051826 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:44.051880 (+ 54us) server_negotiation.cc:407] Connection header received
0812 07:12:44.052625 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:44.052651 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:44.053061 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:44.053533 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:44.054414 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:44.055572 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:44.056461 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:44.056965 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:44.057163 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:44.060100 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:44.060106 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:44.060122 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:44.060155 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:44.062473 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:44.063094 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:44.063101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:44.063108 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:44.063272 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:44.063643 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:44.063646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:44.063648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:44.064282 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:44.064518 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:44.064947 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 07:12:44.065342 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":165,"threads_started":1}
I20260812 07:12:44.070022 23931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d57d02677e428194452a2510737749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45977 } }
I20260812 07:12:44.070410 23931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:44.070415 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:44.070508 23931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d57d02677e428194452a2510737749, State: Initialized, Role: FOLLOWER
I20260812 07:12:44.071048 23931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d57d02677e428194452a2510737749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45977 } }
I20260812 07:12:44.071233 23931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:44.071334 23931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:44.071455 23931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:44.072693 23931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d57d02677e428194452a2510737749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45977 } }
I20260812 07:12:44.073167 23931 leader_election.cc:302] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62d57d02677e428194452a2510737749; no voters:
I20260812 07:12:44.073612 23931 leader_election.cc:288] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:44.073727 23942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:44.074095 23942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [term 1 LEADER]: Becoming Leader. State: Replica: 62d57d02677e428194452a2510737749, State: Running, Role: LEADER
I20260812 07:12:44.074609 23942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d57d02677e428194452a2510737749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45977 } }
I20260812 07:12:44.075424 23931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:44.076628 23944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62d57d02677e428194452a2510737749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d57d02677e428194452a2510737749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45977 } } }
I20260812 07:12:44.076785 23944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:44.077179 23943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62d57d02677e428194452a2510737749. Latest consensus state: current_term: 1 leader_uuid: "62d57d02677e428194452a2510737749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d57d02677e428194452a2510737749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45977 } } }
I20260812 07:12:44.077476 23943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:44.078200 23953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:44.081851 23953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:44.084205 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:44.073038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51789 (local address 127.0.0.1:45977)
0812 07:12:44.073200 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:44.073205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:44.073470 (+ 265us) server_negotiation.cc:407] Connection header received
0812 07:12:44.073630 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:44.073634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:44.073701 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:44.073819 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:44.075034 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:44.075951 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:44.077467 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:44.077861 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:44.077976 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:44.080059 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:44.080064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:44.080067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:44.080096 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:44.082041 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:44.082625 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:44.082629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:44.082631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:44.082764 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:44.083120 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:44.083123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:44.083125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:44.083386 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:44.083533 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:44.083872 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 07:12:44.084011 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:12:44.089171 23953 catalog_manager.cc:1347] Generated new cluster ID: 2a0dc6c7b1fe4d16bb569e1cad47e102
I20260812 07:12:44.089370 23953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:44.106377 23953 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:44.107563 23953 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:44.114519 23953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62d57d02677e428194452a2510737749: Generated new TSK 0
I20260812 07:12:44.115823 23953 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:44.124395 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 23854
2026-08-12T07:12:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1630 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:44 dist-test-slave-csg5 krb5kdc[23969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:44 dist-test-slave-csg5 krb5kdc[23969](info): set up 2 sockets
Aug 12 07:12:44 dist-test-slave-csg5 krb5kdc[23969](info): commencing operation
krb5kdc: starting...
W20260812 07:12:45.307233 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:45 dist-test-slave-csg5 krb5kdc[23969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:45.464452 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37725
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:45.593983 23987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:45.594440 23987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:45.594537 23987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:45.599844 23987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:45.600003 23987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:45.600045 23987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:45.600088 23987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:45.600123 23987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:45.607724 23987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:45.609244 23987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:45.610523 23987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:45.622416 23992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:45.622416 23997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:45.622527 23987 server_base.cc:1034] running on GCE node
W20260812 07:12:45.622329 23995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:45.623386 23987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:45.624802 23987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:45.626026 23987 hybrid_clock.cc:648] HybridClock initialized: now 1786518765625993 us; error 50 us; skew 500 ppm
Aug 12 07:12:45 dist-test-slave-csg5 krb5kdc[23969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518765, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:45.629666 23987 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:45.630903 23987 webserver.cc:460] Webserver started at http://127.0.0.1:43677/ using document root <none> and password file <none>
I20260812 07:12:45.631510 23987 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:45.631577 23987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:45.631771 23987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:45.634023 23987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "1ae76224e4f944fe9747b50120f848d8"
format_stamp: "Formatted at 2026-08-12 07:12:45 on dist-test-slave-csg5"
server_key: "7da0588a34def9359804a5fa2cf8ab72"
server_key_iv: "b3a4117bfc4d53819c8f08863d35e807"
server_key_version: "encryptionkey@0"
I20260812 07:12:45.634601 23987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "1ae76224e4f944fe9747b50120f848d8"
format_stamp: "Formatted at 2026-08-12 07:12:45 on dist-test-slave-csg5"
server_key: "7da0588a34def9359804a5fa2cf8ab72"
server_key_iv: "b3a4117bfc4d53819c8f08863d35e807"
server_key_version: "encryptionkey@0"
I20260812 07:12:45.639007 23987 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:12:45.641901 24006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:45.643678 23987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:45.643872 23987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "1ae76224e4f944fe9747b50120f848d8"
format_stamp: "Formatted at 2026-08-12 07:12:45 on dist-test-slave-csg5"
server_key: "7da0588a34def9359804a5fa2cf8ab72"
server_key_iv: "b3a4117bfc4d53819c8f08863d35e807"
server_key_version: "encryptionkey@0"
I20260812 07:12:45.644022 23987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:45.662150 23987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:45.666059 23987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:45.666313 23987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:45.675060 23987 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37725
I20260812 07:12:45.675112 24074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37725 every 8 connection(s)
I20260812 07:12:45.676185 23987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:45.679687 24075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:45.685221 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 23987
I20260812 07:12:45.685390 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:45.685708 32369 external_mini_cluster.cc:1428] Setting key 578a72a01ef4d31fb22e8fd006d28158
I20260812 07:12:45.687234 24075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8: Bootstrap starting.
I20260812 07:12:45.689839 24075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:45.690785 24075 log.cc:824] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:45.693667 24075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8: No bootstrap required, opened a new log
I20260812 07:12:45.697878 24075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae76224e4f944fe9747b50120f848d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37725 } }
I20260812 07:12:45.698267 24075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:45.698344 24075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ae76224e4f944fe9747b50120f848d8, State: Initialized, Role: FOLLOWER
I20260812 07:12:45.698922 24075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae76224e4f944fe9747b50120f848d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37725 } }
I20260812 07:12:45.699074 24075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:45.699167 24075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:45.699275 24075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:45.700554 24075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae76224e4f944fe9747b50120f848d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37725 } }
I20260812 07:12:45.700973 24075 leader_election.cc:302] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ae76224e4f944fe9747b50120f848d8; no voters:
I20260812 07:12:45.701231 24078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:45.687413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45279 (local address 127.0.0.1:37725)
0812 07:12:45.688012 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:45.688034 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:45.688075 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:12:45.688868 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:45.688888 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:45.689291 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:45.689628 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:45.690105 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:45.691509 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:45.692956 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:45.693540 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:45.693698 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:45.698682 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:45.698694 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:45.698861 (+ 167us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:45.699487 (+ 626us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:45.699569 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:45.699602 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:45.699703 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:45.699923 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 07:12:45.700190 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":213,"threads_started":1}
I20260812 07:12:45.701361 24075 leader_election.cc:288] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:45.701465 24081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:45.701678 24081 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [term 1 LEADER]: Becoming Leader. State: Replica: 1ae76224e4f944fe9747b50120f848d8, State: Running, Role: LEADER
I20260812 07:12:45.702302 24081 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae76224e4f944fe9747b50120f848d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37725 } }
I20260812 07:12:45.703025 24075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:45.704377 24082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ae76224e4f944fe9747b50120f848d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae76224e4f944fe9747b50120f848d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37725 } } }
I20260812 07:12:45.704533 24082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:45.704897 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:45.704931 24084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ae76224e4f944fe9747b50120f848d8. Latest consensus state: current_term: 1 leader_uuid: "1ae76224e4f944fe9747b50120f848d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae76224e4f944fe9747b50120f848d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37725 } } }
I20260812 07:12:45.705035 24084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:45.705415 24092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:45.710157 24092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:45.713639 24078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:45.708304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53411 (local address 127.0.0.1:37725)
0812 07:12:45.708440 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:45.708445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:45.708805 (+ 360us) server_negotiation.cc:407] Connection header received
0812 07:12:45.709007 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:45.709011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:45.709071 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:45.709179 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:45.709547 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:45.710310 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:45.711572 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:45.712010 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:45.712144 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:45.713124 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:45.713129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:45.713138 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:45.713154 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:45.713184 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:45.713196 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:45.713279 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:45.713399 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 07:12:45.713463 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:12:45.718441 24092 catalog_manager.cc:1347] Generated new cluster ID: a35ab337724a4d3aba3a0ae98bb347a7
I20260812 07:12:45.718568 24092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:45.730865 24092 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:45.732124 24092 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:45.744805 24092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ae76224e4f944fe9747b50120f848d8: Generated new TSK 0
I20260812 07:12:45.745721 24092 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:45.753760 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 23987
2026-08-12T07:12:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1628 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:45 dist-test-slave-csg5 krb5kdc[24113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:45 dist-test-slave-csg5 krb5kdc[24113](info): set up 2 sockets
Aug 12 07:12:45 dist-test-slave-csg5 krb5kdc[24113](info): commencing operation
krb5kdc: starting...
W20260812 07:12:46.994405 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:47 dist-test-slave-csg5 krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:47.152208 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37751
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:47.280826 24128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:47.281157 24128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:47.281203 24128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:47.285431 24128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:47.285523 24128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:47.285544 24128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:47.285562 24128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:47.285578 24128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:47.291961 24128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:47.293785 24128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:47.295384 24128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:47.306473 24135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:47.306569 24128 server_base.cc:1034] running on GCE node
W20260812 07:12:47.306490 24136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:47.306596 24138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:47.307614 24128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:47.308950 24128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:47.310148 24128 hybrid_clock.cc:648] HybridClock initialized: now 1786518767310103 us; error 64 us; skew 500 ppm
Aug 12 07:12:47 dist-test-slave-csg5 krb5kdc[24113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518767, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:47.313791 24128 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:47.315096 24128 webserver.cc:460] Webserver started at http://127.0.0.1:36811/ using document root <none> and password file <none>
I20260812 07:12:47.315825 24128 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:47.315953 24128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:47.316293 24128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:47.318735 24128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "90c9b40970cd4b10a768b58faacf56a6"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-csg5"
server_key: "fb84fb33784cd9e807bc13317a2be6b6"
server_key_iv: "e4ca8700be6ed64bbb1691ebdd19ea8d"
server_key_version: "encryptionkey@0"
I20260812 07:12:47.319407 24128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "90c9b40970cd4b10a768b58faacf56a6"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-csg5"
server_key: "fb84fb33784cd9e807bc13317a2be6b6"
server_key_iv: "e4ca8700be6ed64bbb1691ebdd19ea8d"
server_key_version: "encryptionkey@0"
I20260812 07:12:47.323956 24128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:12:47.326946 24150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:47.328606 24128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:12:47.328800 24128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "90c9b40970cd4b10a768b58faacf56a6"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-csg5"
server_key: "fb84fb33784cd9e807bc13317a2be6b6"
server_key_iv: "e4ca8700be6ed64bbb1691ebdd19ea8d"
server_key_version: "encryptionkey@0"
I20260812 07:12:47.328951 24128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:47.334930 24128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:47.339025 24128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:47.339409 24128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:47.348520 24128 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37751
I20260812 07:12:47.348845 24211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37751 every 8 connection(s)
I20260812 07:12:47.349701 24128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:47.351512 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 24128
I20260812 07:12:47.351692 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:47.352011 32369 external_mini_cluster.cc:1428] Setting key d1aed1195266f3c22d96391b5001cc9c
I20260812 07:12:47.354823 24212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:47.358253 24215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:47.353782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42347 (local address 127.0.0.1:37751)
0812 07:12:47.354266 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:47.354283 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:47.354327 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:12:47.355130 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:47.355152 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:47.355494 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:47.355860 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:47.356861 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:47.356865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:47.356877 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:47.356889 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:47.356947 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:47.356968 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:47.357091 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:47.357229 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 07:12:47.357424 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":139,"threads_started":1}
I20260812 07:12:47.361752 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:47.363121 24212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6: Bootstrap starting.
I20260812 07:12:47.366374 24212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:47.366812 24215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:47.364587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36643 (local address 127.0.0.1:37751)
0812 07:12:47.364737 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:47.364745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:47.365024 (+ 279us) server_negotiation.cc:407] Connection header received
0812 07:12:47.365229 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:47.365232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:47.365282 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:47.365381 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:47.366325 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:47.366328 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:47.366331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:47.366343 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:47.366379 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:47.366390 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:47.366489 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:47.366609 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 07:12:47.366675 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:12:47.367566 24212 log.cc:824] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:47.370116 24212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6: No bootstrap required, opened a new log
I20260812 07:12:47.374688 24212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9b40970cd4b10a768b58faacf56a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } }
I20260812 07:12:47.375016 24212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:47.375085 24212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c9b40970cd4b10a768b58faacf56a6, State: Initialized, Role: FOLLOWER
I20260812 07:12:47.375609 24212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9b40970cd4b10a768b58faacf56a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } }
I20260812 07:12:47.375756 24212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:47.375856 24212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:47.375995 24212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:47.377355 24212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9b40970cd4b10a768b58faacf56a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } }
I20260812 07:12:47.377767 24212 leader_election.cc:302] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90c9b40970cd4b10a768b58faacf56a6; no voters:
I20260812 07:12:47.378173 24212 leader_election.cc:288] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:47.378346 24223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:47.378654 24223 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [term 1 LEADER]: Becoming Leader. State: Replica: 90c9b40970cd4b10a768b58faacf56a6, State: Running, Role: LEADER
I20260812 07:12:47.379129 24223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9b40970cd4b10a768b58faacf56a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } }
I20260812 07:12:47.379729 24212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:47.381081 24225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90c9b40970cd4b10a768b58faacf56a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9b40970cd4b10a768b58faacf56a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } } }
I20260812 07:12:47.381232 24225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:47.381608 24226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90c9b40970cd4b10a768b58faacf56a6. Latest consensus state: current_term: 1 leader_uuid: "90c9b40970cd4b10a768b58faacf56a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9b40970cd4b10a768b58faacf56a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } } }
I20260812 07:12:47.381672 24234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:47.381789 24226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:47.385653 24234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:47.392827 24234 catalog_manager.cc:1347] Generated new cluster ID: 46a652d90b1d4275a95c92a7da75d5d4
I20260812 07:12:47.392933 24234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:47.408263 24234 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:47.408782 24234 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:47.414188 24234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90c9b40970cd4b10a768b58faacf56a6: Generated new TSK 0
I20260812 07:12:47.415095 24234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:47.427987 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 24128
2026-08-12T07:12:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1674 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:47 dist-test-slave-csg5 krb5kdc[24247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:47 dist-test-slave-csg5 krb5kdc[24247](info): set up 2 sockets
Aug 12 07:12:47 dist-test-slave-csg5 krb5kdc[24247](info): commencing operation
krb5kdc: starting...
W20260812 07:12:48.630286 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.009s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:48 dist-test-slave-csg5 krb5kdc[24247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:48.779305 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41731
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:48.901432 24263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:48.901861 24263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:48.901947 24263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:48.906606 24263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:48.906806 24263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:48.906845 24263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:48.906877 24263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:48.906913 24263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:48.913151 24263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:48.914472 24263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:48.915721 24263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:12:48.925592 24263 server_base.cc:1034] running on GCE node
W20260812 07:12:48.925513 24268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:48.925547 24269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:48.925530 24273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:48.926560 24263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:48.927913 24263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:48.929229 24263 hybrid_clock.cc:648] HybridClock initialized: now 1786518768929197 us; error 49 us; skew 500 ppm
Aug 12 07:12:48 dist-test-slave-csg5 krb5kdc[24247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518768, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:48.932770 24263 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:48.933851 24263 webserver.cc:460] Webserver started at http://127.0.0.1:44335/ using document root <none> and password file <none>
I20260812 07:12:48.934489 24263 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:48.934561 24263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:48.934859 24263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:48.937341 24263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "0aab48390e15438cadc45ce6322777bf"
format_stamp: "Formatted at 2026-08-12 07:12:48 on dist-test-slave-csg5"
server_key: "09a150cd09ec6292cecf6f345f3224b1"
server_key_iv: "d3daa222ed1a389bda288114e5454e16"
server_key_version: "encryptionkey@0"
I20260812 07:12:48.938023 24263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "0aab48390e15438cadc45ce6322777bf"
format_stamp: "Formatted at 2026-08-12 07:12:48 on dist-test-slave-csg5"
server_key: "09a150cd09ec6292cecf6f345f3224b1"
server_key_iv: "d3daa222ed1a389bda288114e5454e16"
server_key_version: "encryptionkey@0"
I20260812 07:12:48.942564 24263 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:12:48.946228 24281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:48.948379 24263 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260812 07:12:48.948561 24263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "0aab48390e15438cadc45ce6322777bf"
format_stamp: "Formatted at 2026-08-12 07:12:48 on dist-test-slave-csg5"
server_key: "09a150cd09ec6292cecf6f345f3224b1"
server_key_iv: "d3daa222ed1a389bda288114e5454e16"
server_key_version: "encryptionkey@0"
I20260812 07:12:48.948729 24263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:48.970266 24263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:48.974184 24263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:48.974447 24263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:48.982929 24263 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41731
I20260812 07:12:48.983019 24343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41731 every 8 connection(s)
I20260812 07:12:48.984086 24263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:48.987320 24344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:48.988539 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 24263
I20260812 07:12:48.988713 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:48.989020 32369 external_mini_cluster.cc:1428] Setting key 238b7ae723c648b8e4e5451e75180e9b
I20260812 07:12:48.994213 24344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf: Bootstrap starting.
I20260812 07:12:48.997077 24344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:48.998077 24344 log.cc:824] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:12:48 dist-test-slave-csg5 krb5kdc[24247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:12:49.000747 24344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf: No bootstrap required, opened a new log
I20260812 07:12:49.004786 24344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aab48390e15438cadc45ce6322777bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41731 } }
I20260812 07:12:49.005087 24344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:49.005189 24344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aab48390e15438cadc45ce6322777bf, State: Initialized, Role: FOLLOWER
I20260812 07:12:49.005677 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.990790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:37015 (local address 127.0.0.1:41731)
0812 07:12:48.991241 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:48.991259 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:48.991297 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:12:48.991968 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:48.991988 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:48.992366 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:48.992660 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:48.993692 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.994867 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.996085 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.996658 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.996824 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:49.000249 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:49.000255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:49.000271 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:49.000302 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:49.002160 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:49.002803 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:49.002813 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:49.002819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:49.002942 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:49.003377 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:49.003381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:49.003383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:49.003839 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:49.004130 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:49.004440 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 07:12:49.004691 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":136,"threads_started":1}
I20260812 07:12:49.005658 24344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aab48390e15438cadc45ce6322777bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41731 } }
I20260812 07:12:49.005839 24344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:49.005920 24344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:49.006050 24344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:49.007339 24344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aab48390e15438cadc45ce6322777bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41731 } }
I20260812 07:12:49.007732 24344 leader_election.cc:302] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0aab48390e15438cadc45ce6322777bf; no voters:
I20260812 07:12:49.008158 24344 leader_election.cc:288] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:49.008421 24349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:49.008764 24349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [term 1 LEADER]: Becoming Leader. State: Replica: 0aab48390e15438cadc45ce6322777bf, State: Running, Role: LEADER
I20260812 07:12:49.009179 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:49.009188 24349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aab48390e15438cadc45ce6322777bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41731 } }
I20260812 07:12:49.009581 24344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:49.010510 24349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0aab48390e15438cadc45ce6322777bf. Latest consensus state: current_term: 1 leader_uuid: "0aab48390e15438cadc45ce6322777bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aab48390e15438cadc45ce6322777bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41731 } } }
I20260812 07:12:49.010645 24349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:49.011189 24363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:49.012039 24350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0aab48390e15438cadc45ce6322777bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aab48390e15438cadc45ce6322777bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41731 } } }
I20260812 07:12:49.012204 24350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:49.015846 24363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:49.022698 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:49.011837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:38989 (local address 127.0.0.1:41731)
0812 07:12:49.011956 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:49.011961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:49.012486 (+ 525us) server_negotiation.cc:407] Connection header received
0812 07:12:49.012628 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:49.012632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:49.012688 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:49.012781 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:49.013627 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:49.014381 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:49.015626 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:49.015989 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:49.016119 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:49.018092 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:49.018096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:49.018099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:49.018132 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:49.020352 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:49.020968 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:49.020972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:49.020974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:49.021078 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:49.021599 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:49.021603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:49.021605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:49.021869 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:49.022033 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:49.022374 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 07:12:49.022492 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 07:12:49.024030 24363 catalog_manager.cc:1347] Generated new cluster ID: e562c054552f40d99c9c3908f308fc58
I20260812 07:12:49.024168 24363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:49.045944 24363 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:49.047284 24363 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:49.056098 24363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0aab48390e15438cadc45ce6322777bf: Generated new TSK 0
I20260812 07:12:49.057112 24363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:49.063503 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 24263
W20260812 07:12:49.075150 24245 connection.cc:535] client connection to 127.0.0.1:41731 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41731): Connection reset by peer (error 104)
2026-08-12T07:12:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1636 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:49 dist-test-slave-csg5 krb5kdc[24380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:49 dist-test-slave-csg5 krb5kdc[24380](info): set up 2 sockets
Aug 12 07:12:49 dist-test-slave-csg5 krb5kdc[24380](info): commencing operation
krb5kdc: starting...
W20260812 07:12:50.295568 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:50 dist-test-slave-csg5 krb5kdc[24380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:50.475549 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:50.605446 24395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:50.605784 24395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:50.605830 24395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:50.610117 24395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:50.610209 24395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:50.610229 24395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:50.610247 24395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:50.610263 24395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:50.616107 24395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:50.617334 24395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:50.618420 24395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:50.627645 24402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:50.627882 24404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:50.628072 24395 server_base.cc:1034] running on GCE node
W20260812 07:12:50.627893 24401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:50.628952 24395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:50.630288 24395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:50.631518 24395 hybrid_clock.cc:648] HybridClock initialized: now 1786518770631488 us; error 53 us; skew 500 ppm
Aug 12 07:12:50 dist-test-slave-csg5 krb5kdc[24380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518770, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:50.634992 24395 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:50.636009 24395 webserver.cc:460] Webserver started at http://127.0.0.1:45863/ using document root <none> and password file <none>
I20260812 07:12:50.636713 24395 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:50.636816 24395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:50.637079 24395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:50.639451 24395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "03db48437a5f4c79a07438d39bf2cca5"
format_stamp: "Formatted at 2026-08-12 07:12:50 on dist-test-slave-csg5"
server_key: "2c036c58a8edc72f587770723fa82818"
server_key_iv: "be1d428c7cf45fb27c2253cc9171b368"
server_key_version: "encryptionkey@0"
I20260812 07:12:50.640077 24395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "03db48437a5f4c79a07438d39bf2cca5"
format_stamp: "Formatted at 2026-08-12 07:12:50 on dist-test-slave-csg5"
server_key: "2c036c58a8edc72f587770723fa82818"
server_key_iv: "be1d428c7cf45fb27c2253cc9171b368"
server_key_version: "encryptionkey@0"
I20260812 07:12:50.644277 24395 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:50.647408 24411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:50.649032 24395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:12:50.649232 24395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "03db48437a5f4c79a07438d39bf2cca5"
format_stamp: "Formatted at 2026-08-12 07:12:50 on dist-test-slave-csg5"
server_key: "2c036c58a8edc72f587770723fa82818"
server_key_iv: "be1d428c7cf45fb27c2253cc9171b368"
server_key_version: "encryptionkey@0"
I20260812 07:12:50.649444 24395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:50.678525 24395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:50.682950 24395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:50.683243 24395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:50.692034 24395 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35109
I20260812 07:12:50.692104 24469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35109 every 8 connection(s)
I20260812 07:12:50.693199 24395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:50.695751 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 24395
I20260812 07:12:50.695924 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:50.696287 32369 external_mini_cluster.cc:1428] Setting key 0629467282c7ed05725d5a5815820232
I20260812 07:12:50.697167 24470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:50.704638 24470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5: Bootstrap starting.
I20260812 07:12:50.707504 24470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:50.708350 24470 log.cc:824] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:50.711169 24470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5: No bootstrap required, opened a new log
I20260812 07:12:50.712786 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:50.697985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:41703 (local address 127.0.0.1:35109)
0812 07:12:50.698579 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:50.698596 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:50.698638 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:12:50.699462 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:50.699482 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:50.699897 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:50.700314 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:50.700776 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.701919 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.703042 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.703601 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.703813 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:50.710345 (+ 6532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:50.710351 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:50.710367 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:50.710941 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:50.711013 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:50.711041 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:50.711220 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:50.711504 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 07:12:50.711675 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":187,"threads_started":1}
I20260812 07:12:50.716703 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:50.716643 24470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db48437a5f4c79a07438d39bf2cca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35109 } }
I20260812 07:12:50.716993 24470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:50.717106 24470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03db48437a5f4c79a07438d39bf2cca5, State: Initialized, Role: FOLLOWER
I20260812 07:12:50.717635 24470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db48437a5f4c79a07438d39bf2cca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35109 } }
I20260812 07:12:50.717782 24470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:50.717877 24470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:50.718019 24470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:50.719373 24470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db48437a5f4c79a07438d39bf2cca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35109 } }
I20260812 07:12:50.719890 24470 leader_election.cc:302] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03db48437a5f4c79a07438d39bf2cca5; no voters:
I20260812 07:12:50.720357 24470 leader_election.cc:288] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:50.720441 24482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:50.720693 24482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [term 1 LEADER]: Becoming Leader. State: Replica: 03db48437a5f4c79a07438d39bf2cca5, State: Running, Role: LEADER
I20260812 07:12:50.721122 24482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db48437a5f4c79a07438d39bf2cca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35109 } }
I20260812 07:12:50.721963 24470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:50.723111 24484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03db48437a5f4c79a07438d39bf2cca5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db48437a5f4c79a07438d39bf2cca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35109 } } }
I20260812 07:12:50.723652 24484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:50.724079 24483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03db48437a5f4c79a07438d39bf2cca5. Latest consensus state: current_term: 1 leader_uuid: "03db48437a5f4c79a07438d39bf2cca5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db48437a5f4c79a07438d39bf2cca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35109 } } }
I20260812 07:12:50.724278 24483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:50.724650 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:50.719365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:57481 (local address 127.0.0.1:35109)
0812 07:12:50.719522 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:50.719527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:50.719792 (+ 265us) server_negotiation.cc:407] Connection header received
0812 07:12:50.719976 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:50.719979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:50.720026 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:50.720123 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:50.720432 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.721168 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.722180 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.722468 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.722568 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:50.723827 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:50.723832 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:50.723834 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:50.723849 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:50.723882 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:50.723897 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:50.723979 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:50.724462 (+ 483us) server_negotiation.cc:299] Negotiation successful
0812 07:12:50.724507 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 07:12:50.728066 24505 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:12:50.728170 24505 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:12:50.728272 24493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:50.729218 24493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:50.736114 24493 catalog_manager.cc:1347] Generated new cluster ID: ec77cc799a4747718fb9fa67ad7a32ec
I20260812 07:12:50.736223 24493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:50.756527 24493 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:50.757552 24493 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:50.763757 24493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03db48437a5f4c79a07438d39bf2cca5: Generated new TSK 0
I20260812 07:12:50.764576 24493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:50.774583 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 24395
W20260812 07:12:50.783998 24377 connection.cc:535] client connection to 127.0.0.1:35109 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35109): Connection reset by peer (error 104)
2026-08-12T07:12:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1708 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:50 dist-test-slave-csg5 krb5kdc[24511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:50 dist-test-slave-csg5 krb5kdc[24511](info): set up 2 sockets
Aug 12 07:12:50 dist-test-slave-csg5 krb5kdc[24511](info): commencing operation
krb5kdc: starting...
W20260812 07:12:52.010063 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:52 dist-test-slave-csg5 krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:52.170994 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:52.316890 24528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:52.317313 24528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:52.317417 24528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:52.321887 24528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:52.322009 24528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:52.322049 24528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:52.322119 24528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:52.322180 24528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:52.328096 24528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:52.329371 24528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:52.330497 24528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:52.340317 24537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:52.340328 24534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:52.340341 24535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:52.340566 24528 server_base.cc:1034] running on GCE node
I20260812 07:12:52.341246 24528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:52.342546 24528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:52.343762 24528 hybrid_clock.cc:648] HybridClock initialized: now 1786518772343719 us; error 40 us; skew 500 ppm
Aug 12 07:12:52 dist-test-slave-csg5 krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518772, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:52.347266 24528 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:52.348605 24528 webserver.cc:460] Webserver started at http://127.0.0.1:42523/ using document root <none> and password file <none>
I20260812 07:12:52.349472 24528 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:52.349586 24528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:52.349879 24528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:52.353350 24528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "ad0b2400d57647ac9740afd6c88f6f48"
format_stamp: "Formatted at 2026-08-12 07:12:52 on dist-test-slave-csg5"
server_key: "b8f435ec16b676e22aa5111013005037"
server_key_iv: "9e9f890a9cada9fc1cd192e9eb352758"
server_key_version: "encryptionkey@0"
I20260812 07:12:52.354344 24528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "ad0b2400d57647ac9740afd6c88f6f48"
format_stamp: "Formatted at 2026-08-12 07:12:52 on dist-test-slave-csg5"
server_key: "b8f435ec16b676e22aa5111013005037"
server_key_iv: "9e9f890a9cada9fc1cd192e9eb352758"
server_key_version: "encryptionkey@0"
I20260812 07:12:52.360599 24528 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260812 07:12:52.364116 24544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:52.365928 24528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:12:52.366072 24528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "ad0b2400d57647ac9740afd6c88f6f48"
format_stamp: "Formatted at 2026-08-12 07:12:52 on dist-test-slave-csg5"
server_key: "b8f435ec16b676e22aa5111013005037"
server_key_iv: "9e9f890a9cada9fc1cd192e9eb352758"
server_key_version: "encryptionkey@0"
I20260812 07:12:52.366210 24528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:52.371308 24528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:52.375326 24528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:52.375530 24528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:52.383889 24528 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40461
I20260812 07:12:52.384001 24605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40461 every 8 connection(s)
I20260812 07:12:52.385100 24528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:52.388648 24608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:52.391049 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 24528
I20260812 07:12:52.391299 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:52.391619 32369 external_mini_cluster.cc:1428] Setting key 92de1fc63c9c5cc8008f3b3a392a7a1d
I20260812 07:12:52.395988 24608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48: Bootstrap starting.
I20260812 07:12:52.398260 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:52.393337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:60713 (local address 127.0.0.1:40461)
0812 07:12:52.393924 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:52.393940 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:52.393982 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:12:52.394995 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:52.395018 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:52.395468 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:52.396053 (+ 585us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:52.396947 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:52.396952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:52.396964 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:52.396974 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:52.397034 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:52.397055 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:52.397228 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:52.397314 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 07:12:52.397493 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":180,"threads_started":1}
I20260812 07:12:52.399133 24608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:52.400096 24608 log.cc:824] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:52.401561 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:52.402783 24608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48: No bootstrap required, opened a new log
I20260812 07:12:52.405972 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:52.403936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:52039 (local address 127.0.0.1:40461)
0812 07:12:52.404085 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:52.404092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:52.404248 (+ 156us) server_negotiation.cc:407] Connection header received
0812 07:12:52.404414 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:52.404417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:52.404449 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:52.404520 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:52.405399 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:52.405402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:52.405404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:52.405416 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:52.405443 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:52.405454 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:52.405538 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:52.405725 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 07:12:52.405805 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:12:52.408082 24608 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b2400d57647ac9740afd6c88f6f48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } }
I20260812 07:12:52.408403 24608 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:52.408538 24608 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad0b2400d57647ac9740afd6c88f6f48, State: Initialized, Role: FOLLOWER
I20260812 07:12:52.409142 24608 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b2400d57647ac9740afd6c88f6f48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } }
I20260812 07:12:52.409332 24608 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:52.409436 24608 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:52.409554 24608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:52.410794 24608 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b2400d57647ac9740afd6c88f6f48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } }
I20260812 07:12:52.411201 24608 leader_election.cc:302] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad0b2400d57647ac9740afd6c88f6f48; no voters:
I20260812 07:12:52.411621 24608 leader_election.cc:288] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:52.411782 24618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:52.412083 24618 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [term 1 LEADER]: Becoming Leader. State: Replica: ad0b2400d57647ac9740afd6c88f6f48, State: Running, Role: LEADER
I20260812 07:12:52.412619 24618 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b2400d57647ac9740afd6c88f6f48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } }
I20260812 07:12:52.413255 24608 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:52.414844 24621 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad0b2400d57647ac9740afd6c88f6f48. Latest consensus state: current_term: 1 leader_uuid: "ad0b2400d57647ac9740afd6c88f6f48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b2400d57647ac9740afd6c88f6f48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } } }
I20260812 07:12:52.414986 24621 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:52.415263 24619 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad0b2400d57647ac9740afd6c88f6f48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b2400d57647ac9740afd6c88f6f48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } } }
I20260812 07:12:52.415436 24619 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48 [sys.catalog]: This master's current role is: LEADER
W20260812 07:12:52.418984 24635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:12:52.419076 24635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:12:52.419165 24628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:52.419917 24628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:52.425879 24628 catalog_manager.cc:1347] Generated new cluster ID: 27b95562607a4050a49fb0777bcf6c35
I20260812 07:12:52.425974 24628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:52.438995 24628 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:52.439445 24628 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:52.449433 24628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad0b2400d57647ac9740afd6c88f6f48: Generated new TSK 0
I20260812 07:12:52.450592 24628 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:52.463501 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 24528
2026-08-12T07:12:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1691 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:52 dist-test-slave-csg5 krb5kdc[24642](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:52 dist-test-slave-csg5 krb5kdc[24642](info): set up 2 sockets
Aug 12 07:12:52 dist-test-slave-csg5 krb5kdc[24642](info): commencing operation
krb5kdc: starting...
W20260812 07:12:53.642498 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:53 dist-test-slave-csg5 krb5kdc[24642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:53.800773 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37905
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:53.924069 24656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:53.924428 24656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:53.924521 24656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:53.928949 24656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:53.929067 24656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:53.929106 24656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:53.929147 24656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:53.929184 24656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:53.935190 24656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37905
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:53.936479 24656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:53.937610 24656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:12:53.947105 24656 server_base.cc:1034] running on GCE node
W20260812 07:12:53.947129 24663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:53.947294 24665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:53.947446 24662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:53.948113 24656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:53.949314 24656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:53.950527 24656 hybrid_clock.cc:648] HybridClock initialized: now 1786518773950512 us; error 43 us; skew 500 ppm
Aug 12 07:12:53 dist-test-slave-csg5 krb5kdc[24642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:53.953691 24656 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:53.954854 24656 webserver.cc:460] Webserver started at http://127.0.0.1:39107/ using document root <none> and password file <none>
I20260812 07:12:53.955430 24656 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:53.955495 24656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:53.955677 24656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:53.957890 24656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "432546196281497fbeb7ff4a73d84138"
format_stamp: "Formatted at 2026-08-12 07:12:53 on dist-test-slave-csg5"
server_key: "920f950f8bc59830390056cb19f3eec8"
server_key_iv: "26000a112c4f25d2d65c7ec53c48a0b3"
server_key_version: "encryptionkey@0"
I20260812 07:12:53.958456 24656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "432546196281497fbeb7ff4a73d84138"
format_stamp: "Formatted at 2026-08-12 07:12:53 on dist-test-slave-csg5"
server_key: "920f950f8bc59830390056cb19f3eec8"
server_key_iv: "26000a112c4f25d2d65c7ec53c48a0b3"
server_key_version: "encryptionkey@0"
I20260812 07:12:53.963019 24656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:53.966215 24674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:53.968295 24656 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:12:53.968490 24656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "432546196281497fbeb7ff4a73d84138"
format_stamp: "Formatted at 2026-08-12 07:12:53 on dist-test-slave-csg5"
server_key: "920f950f8bc59830390056cb19f3eec8"
server_key_iv: "26000a112c4f25d2d65c7ec53c48a0b3"
server_key_version: "encryptionkey@0"
I20260812 07:12:53.968672 24656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:53.987653 24656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:53.991911 24656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:53.992190 24656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:54.001348 24656 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37905
I20260812 07:12:54.001396 24734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37905 every 8 connection(s)
I20260812 07:12:54.002562 24656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:54.006016 24735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:54.010959 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 24656
I20260812 07:12:54.011102 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:54.011543 32369 external_mini_cluster.cc:1428] Setting key b825bf25a1efb21a132a7ce133d9c4e2
I20260812 07:12:54.013468 24735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138: Bootstrap starting.
I20260812 07:12:54.016868 24735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:54.018010 24735 log.cc:824] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:54.021136 24735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138: No bootstrap required, opened a new log
Aug 12 07:12:54 dist-test-slave-csg5 krb5kdc[24642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:12:54.026484 24735 raft_consensus.cc:348] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432546196281497fbeb7ff4a73d84138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } }
I20260812 07:12:54.026889 24735 raft_consensus.cc:374] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:54.026963 24735 raft_consensus.cc:729] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432546196281497fbeb7ff4a73d84138, State: Initialized, Role: FOLLOWER
I20260812 07:12:54.027487 24735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432546196281497fbeb7ff4a73d84138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } }
I20260812 07:12:54.027695 24735 raft_consensus.cc:388] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:54.027771 24735 raft_consensus.cc:482] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:54.027894 24735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:54.029371 24735 raft_consensus.cc:504] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432546196281497fbeb7ff4a73d84138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } }
I20260812 07:12:54.029798 24735 leader_election.cc:302] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 432546196281497fbeb7ff4a73d84138; no voters:
I20260812 07:12:54.030205 24735 leader_election.cc:288] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:54.030471 24740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:54.030802 24740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [term 1 LEADER]: Becoming Leader. State: Replica: 432546196281497fbeb7ff4a73d84138, State: Running, Role: LEADER
I20260812 07:12:54.031132 24738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:54.013484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:50889 (local address 127.0.0.1:37905)
0812 07:12:54.014077 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:54.014098 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:54.014147 (+ 49us) server_negotiation.cc:407] Connection header received
0812 07:12:54.015240 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:54.015270 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:54.015805 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:54.016249 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:54.017255 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:54.018931 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:54.020130 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:54.020913 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:54.021105 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:54.024241 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:54.024249 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:54.024277 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:54.024330 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:54.026828 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:54.027692 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:54.027699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:54.027707 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:54.027884 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:54.028342 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:54.028346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:54.028348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:54.028959 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:54.029227 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:54.029576 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 07:12:54.029864 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":216,"threads_started":1}
I20260812 07:12:54.031224 24740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432546196281497fbeb7ff4a73d84138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } }
I20260812 07:12:54.031752 24735 sys_catalog.cc:564] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:54.033483 24742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "432546196281497fbeb7ff4a73d84138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432546196281497fbeb7ff4a73d84138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } } }
I20260812 07:12:54.033489 24743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 432546196281497fbeb7ff4a73d84138. Latest consensus state: current_term: 1 leader_uuid: "432546196281497fbeb7ff4a73d84138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432546196281497fbeb7ff4a73d84138" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } } }
I20260812 07:12:54.033646 24742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:54.033650 24743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:54.034591 24752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:54.035696 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:54.038578 24752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:54.046334 24752 catalog_manager.cc:1347] Generated new cluster ID: b6f43857dfca4b6f8b20d011ea505d56
I20260812 07:12:54.046425 24752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:54.049752 24738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:54.038030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:39661 (local address 127.0.0.1:37905)
0812 07:12:54.038198 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:54.038203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:54.039063 (+ 860us) server_negotiation.cc:407] Connection header received
0812 07:12:54.039357 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:54.039360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:54.039411 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:54.039624 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:54.040449 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:54.041230 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:54.042587 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:54.043004 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:54.043163 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:54.045344 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:54.045349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:54.045352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:54.045386 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:54.047457 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:54.048102 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:54.048107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:54.048112 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:54.048226 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:54.048629 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:54.048633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:54.048635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:54.048907 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:54.049098 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:54.049408 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 07:12:54.049534 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:12:54.058979 24752 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:54.060431 24752 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:54.068259 24752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 432546196281497fbeb7ff4a73d84138: Generated new TSK 0
I20260812 07:12:54.069466 24752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:54.079200 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 24656
W20260812 07:12:54.088541 24639 connection.cc:535] client connection to 127.0.0.1:37905 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37905): Connection reset by peer (error 104)
2026-08-12T07:12:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1613 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:54 dist-test-slave-csg5 krb5kdc[24769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:54 dist-test-slave-csg5 krb5kdc[24769](info): set up 2 sockets
Aug 12 07:12:54 dist-test-slave-csg5 krb5kdc[24769](info): commencing operation
krb5kdc: starting...
W20260812 07:12:55.237087 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.004s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:55 dist-test-slave-csg5 krb5kdc[24769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:55.391387 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:55.522532 24783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:55.522966 24783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:55.523063 24783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:55.527582 24783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:55.527730 24783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:55.527768 24783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:55.527812 24783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:55.527848 24783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:55.534016 24783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37429
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:55.535419 24783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:55.536556 24783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:55.546222 24789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:55.546217 24791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:55.546347 24788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:55.546545 24783 server_base.cc:1034] running on GCE node
I20260812 07:12:55.547466 24783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:55.548769 24783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:55.549976 24783 hybrid_clock.cc:648] HybridClock initialized: now 1786518775549947 us; error 45 us; skew 500 ppm
Aug 12 07:12:55 dist-test-slave-csg5 krb5kdc[24769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:55.553565 24783 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:55.554819 24783 webserver.cc:460] Webserver started at http://127.0.0.1:44279/ using document root <none> and password file <none>
I20260812 07:12:55.555501 24783 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:55.555612 24783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:55.555899 24783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:55.558368 24783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "9394e6b17b50462fbcfdfcc967e31710"
format_stamp: "Formatted at 2026-08-12 07:12:55 on dist-test-slave-csg5"
server_key: "2b9518f1af08f0df777921b09547ad6f"
server_key_iv: "be740909f92e752a6b99ecd06d76e66f"
server_key_version: "encryptionkey@0"
I20260812 07:12:55.559139 24783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "9394e6b17b50462fbcfdfcc967e31710"
format_stamp: "Formatted at 2026-08-12 07:12:55 on dist-test-slave-csg5"
server_key: "2b9518f1af08f0df777921b09547ad6f"
server_key_iv: "be740909f92e752a6b99ecd06d76e66f"
server_key_version: "encryptionkey@0"
I20260812 07:12:55.563982 24783 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:12:55.567283 24798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:55.568935 24783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:12:55.569078 24783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "9394e6b17b50462fbcfdfcc967e31710"
format_stamp: "Formatted at 2026-08-12 07:12:55 on dist-test-slave-csg5"
server_key: "2b9518f1af08f0df777921b09547ad6f"
server_key_iv: "be740909f92e752a6b99ecd06d76e66f"
server_key_version: "encryptionkey@0"
I20260812 07:12:55.569202 24783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:55.593686 24783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:55.598008 24783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:55.598229 24783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:55.607249 24783 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37429
I20260812 07:12:55.607285 24857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37429 every 8 connection(s)
I20260812 07:12:55.608431 24783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:55.611236 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 24783
I20260812 07:12:55.611434 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:55.611739 32369 external_mini_cluster.cc:1428] Setting key 01bf32db8522daf55d530b9abf6d8745
I20260812 07:12:55.611963 24858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:55.619372 24858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710: Bootstrap starting.
I20260812 07:12:55.621987 24858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:55.622352 24861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:55.613338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:39597 (local address 127.0.0.1:37429)
0812 07:12:55.613886 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:55.613906 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:55.613945 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:12:55.615102 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:55.615123 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:55.615509 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:55.615889 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:55.616368 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:55.617393 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:55.618342 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:55.618918 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:55.619103 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:55.620179 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:55.620184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:55.620198 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:55.620616 (+ 418us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:55.620680 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:55.620704 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:55.621069 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:55.621254 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 07:12:55.621429 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":182,"threads_started":1}
I20260812 07:12:55.623059 24858 log.cc:824] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:55.625803 24858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710: No bootstrap required, opened a new log
I20260812 07:12:55.626040 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:55.631520 24858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6b17b50462fbcfdfcc967e31710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37429 } }
I20260812 07:12:55.631907 24858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:55.632004 24858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9394e6b17b50462fbcfdfcc967e31710, State: Initialized, Role: FOLLOWER
I20260812 07:12:55.632579 24858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6b17b50462fbcfdfcc967e31710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37429 } }
I20260812 07:12:55.632740 24858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:55.632819 24858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:55.632956 24858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:55.633862 24861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:55.628573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:48411 (local address 127.0.0.1:37429)
0812 07:12:55.628717 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:55.628721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:55.628983 (+ 262us) server_negotiation.cc:407] Connection header received
0812 07:12:55.629107 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:55.629109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:55.629156 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:55.629226 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:55.629580 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:55.630162 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:55.631673 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:55.632044 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:55.632314 (+ 270us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:55.633292 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:55.633297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:55.633300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:55.633326 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:55.633358 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:55.633371 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:55.633490 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:55.633663 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 07:12:55.633706 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 07:12:55.634318 24858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6b17b50462fbcfdfcc967e31710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37429 } }
I20260812 07:12:55.634826 24858 leader_election.cc:302] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9394e6b17b50462fbcfdfcc967e31710; no voters:
I20260812 07:12:55.635268 24858 leader_election.cc:288] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:55.635576 24869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:55.635855 24869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [term 1 LEADER]: Becoming Leader. State: Replica: 9394e6b17b50462fbcfdfcc967e31710, State: Running, Role: LEADER
I20260812 07:12:55.636214 24869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6b17b50462fbcfdfcc967e31710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37429 } }
I20260812 07:12:55.636845 24858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:55.638206 24871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9394e6b17b50462fbcfdfcc967e31710. Latest consensus state: current_term: 1 leader_uuid: "9394e6b17b50462fbcfdfcc967e31710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6b17b50462fbcfdfcc967e31710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37429 } } }
I20260812 07:12:55.638314 24870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9394e6b17b50462fbcfdfcc967e31710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6b17b50462fbcfdfcc967e31710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37429 } } }
I20260812 07:12:55.638370 24871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:55.638403 24870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:55.638897 24880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:55.642815 24880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:55.649801 24880 catalog_manager.cc:1347] Generated new cluster ID: 5c42ee9e91dd4349bc05fc4a8c35399c
I20260812 07:12:55.649911 24880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:55.658923 24880 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:55.660003 24880 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:55.668408 24880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9394e6b17b50462fbcfdfcc967e31710: Generated new TSK 0
I20260812 07:12:55.669298 24880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:55.676420 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 24783
W20260812 07:12:55.691032 24767 connection.cc:535] client connection to 127.0.0.1:37429 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37429): Connection reset by peer (error 104)
2026-08-12T07:12:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1603 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:55 dist-test-slave-csg5 krb5kdc[24894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:55 dist-test-slave-csg5 krb5kdc[24894](info): set up 2 sockets
Aug 12 07:12:55 dist-test-slave-csg5 krb5kdc[24894](info): commencing operation
krb5kdc: starting...
W20260812 07:12:56.876998 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:56 dist-test-slave-csg5 krb5kdc[24894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:12:57.031311 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:57.155792 24910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:57.156160 24910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:57.156209 24910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:57.160710 24910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:57.160797 24910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:57.160818 24910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:57.160836 24910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:57.160852 24910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:57.166622 24910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46653
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:57.167966 24910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:57.169081 24910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:57.178519 24917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:57.178624 24910 server_base.cc:1034] running on GCE node
W20260812 07:12:57.178519 24919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:57.178550 24916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:57.179550 24910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:57.180900 24910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:57.182152 24910 hybrid_clock.cc:648] HybridClock initialized: now 1786518777182181 us; error 89 us; skew 500 ppm
Aug 12 07:12:57 dist-test-slave-csg5 krb5kdc[24894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518777, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:57.186105 24910 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:12:57.187480 24910 webserver.cc:460] Webserver started at http://127.0.0.1:45811/ using document root <none> and password file <none>
I20260812 07:12:57.188164 24910 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:57.188278 24910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:57.188561 24910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:57.191088 24910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "379c3340e0bb4a489df4e83f882d798a"
format_stamp: "Formatted at 2026-08-12 07:12:57 on dist-test-slave-csg5"
server_key: "d5926205c459b434ee417db43db4bea0"
server_key_iv: "1171864f9ec3db27d75d3604c7b89932"
server_key_version: "encryptionkey@0"
I20260812 07:12:57.191913 24910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "379c3340e0bb4a489df4e83f882d798a"
format_stamp: "Formatted at 2026-08-12 07:12:57 on dist-test-slave-csg5"
server_key: "d5926205c459b434ee417db43db4bea0"
server_key_iv: "1171864f9ec3db27d75d3604c7b89932"
server_key_version: "encryptionkey@0"
I20260812 07:12:57.196487 24910 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:57.199417 24926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:57.201038 24910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:57.201238 24910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "379c3340e0bb4a489df4e83f882d798a"
format_stamp: "Formatted at 2026-08-12 07:12:57 on dist-test-slave-csg5"
server_key: "d5926205c459b434ee417db43db4bea0"
server_key_iv: "1171864f9ec3db27d75d3604c7b89932"
server_key_version: "encryptionkey@0"
I20260812 07:12:57.201393 24910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:57.207243 24910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:57.211100 24910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:57.211349 24910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:57.220662 24910 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46653
I20260812 07:12:57.220678 24989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46653 every 8 connection(s)
I20260812 07:12:57.221868 24910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:57.225329 24990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:57.230307 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 24910
I20260812 07:12:57.230490 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:57.230886 32369 external_mini_cluster.cc:1428] Setting key ffb8482fee739e1ec46b579e179e948a
I20260812 07:12:57.232911 24990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a: Bootstrap starting.
I20260812 07:12:57.235957 24990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:57.236982 24990 log.cc:824] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:57.237634 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:57.232845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:48261 (local address 127.0.0.1:46653)
0812 07:12:57.233374 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:57.233390 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:57.233431 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:12:57.234152 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:57.234172 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:57.234489 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:57.234843 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:57.235888 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:57.235894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:57.235908 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:57.235920 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:57.235986 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:57.236009 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:57.236230 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:57.236415 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 07:12:57.236642 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":206,"threads_started":1}
I20260812 07:12:57.240037 24990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a: No bootstrap required, opened a new log
I20260812 07:12:57.241461 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:57.244645 24990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379c3340e0bb4a489df4e83f882d798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46653 } }
I20260812 07:12:57.244969 24990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:57.245061 24990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379c3340e0bb4a489df4e83f882d798a, State: Initialized, Role: FOLLOWER
I20260812 07:12:57.245693 24990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379c3340e0bb4a489df4e83f882d798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46653 } }
I20260812 07:12:57.245925 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:57.243972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.95:33579 (local address 127.0.0.1:46653)
0812 07:12:57.244113 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:57.244118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:57.244241 (+ 123us) server_negotiation.cc:407] Connection header received
0812 07:12:57.244361 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:57.244364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:57.244408 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:57.244505 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:57.245352 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:57.245355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:57.245357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:12:57.245368 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:57.245398 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:12:57.245409 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:57.245500 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:57.245609 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 07:12:57.245648 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:12:57.245967 24990 raft_consensus.cc:388] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:57.246037 24990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:57.246145 24990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:57.247363 24990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379c3340e0bb4a489df4e83f882d798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46653 } }
I20260812 07:12:57.247761 24990 leader_election.cc:302] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 379c3340e0bb4a489df4e83f882d798a; no voters:
I20260812 07:12:57.248116 24990 leader_election.cc:288] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:57.248282 25001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:57.248555 25001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [term 1 LEADER]: Becoming Leader. State: Replica: 379c3340e0bb4a489df4e83f882d798a, State: Running, Role: LEADER
I20260812 07:12:57.248961 25001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379c3340e0bb4a489df4e83f882d798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46653 } }
I20260812 07:12:57.249544 24990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:57.250963 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 379c3340e0bb4a489df4e83f882d798a. Latest consensus state: current_term: 1 leader_uuid: "379c3340e0bb4a489df4e83f882d798a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379c3340e0bb4a489df4e83f882d798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46653 } } }
I20260812 07:12:57.251116 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:57.251525 25012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:57.251605 25002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "379c3340e0bb4a489df4e83f882d798a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "379c3340e0bb4a489df4e83f882d798a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46653 } } }
I20260812 07:12:57.251888 25002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:57.255156 25012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:57.261664 25012 catalog_manager.cc:1347] Generated new cluster ID: 24493bad569d4cd2b54d4499545c12fd
I20260812 07:12:57.261771 25012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:57.276046 25012 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:57.276464 25012 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:57.283749 25012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 379c3340e0bb4a489df4e83f882d798a: Generated new TSK 0
I20260812 07:12:57.284816 25012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:57.297616 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 24910
2026-08-12T07:12:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1617 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:57 dist-test-slave-csg5 krb5kdc[25024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:57 dist-test-slave-csg5 krb5kdc[25024](info): set up 2 sockets
Aug 12 07:12:57 dist-test-slave-csg5 krb5kdc[25024](info): commencing operation
krb5kdc: starting...
W20260812 07:12:58.472074 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:12:58 dist-test-slave-csg5 krb5kdc[25024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:12:58.628510 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46479
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:32795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:46479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:58.756590 25039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:58.756930 25039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:58.756976 25039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:58.761395 25039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:58.761487 25039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:58.761507 25039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:58.761524 25039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:58.761540 25039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:58.767441 25039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:32795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:46479
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:12:58.768821 25039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:58.770246 25039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:58.780112 25048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:58.780226 25039 server_base.cc:1034] running on GCE node
W20260812 07:12:58.780112 25045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:58.780544 25046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:58.781347 25039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:58.782850 25039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:58.784090 25039 hybrid_clock.cc:648] HybridClock initialized: now 1786518778784096 us; error 114 us; skew 500 ppm
Aug 12 07:12:58 dist-test-slave-csg5 krb5kdc[25024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518778, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:58.787987 25039 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:12:58.789206 25039 webserver.cc:460] Webserver started at http://127.31.156.126:46711/ using document root <none> and password file <none>
I20260812 07:12:58.789886 25039 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:58.789997 25039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:58.790272 25039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:58.792785 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "8ed3b1137b734d5191e8974261160513"
format_stamp: "Formatted at 2026-08-12 07:12:58 on dist-test-slave-csg5"
server_key: "8a591c919b6524d76f89f30ae7279368"
server_key_iv: "5ef4a96e2e1a2239b3cd1fd72afe8a2e"
server_key_version: "encryptionkey@0"
I20260812 07:12:58.793495 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "8ed3b1137b734d5191e8974261160513"
format_stamp: "Formatted at 2026-08-12 07:12:58 on dist-test-slave-csg5"
server_key: "8a591c919b6524d76f89f30ae7279368"
server_key_iv: "5ef4a96e2e1a2239b3cd1fd72afe8a2e"
server_key_version: "encryptionkey@0"
I20260812 07:12:58.798786 25039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260812 07:12:58.801954 25058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:58.803764 25039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:58.803968 25039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "8ed3b1137b734d5191e8974261160513"
format_stamp: "Formatted at 2026-08-12 07:12:58 on dist-test-slave-csg5"
server_key: "8a591c919b6524d76f89f30ae7279368"
server_key_iv: "5ef4a96e2e1a2239b3cd1fd72afe8a2e"
server_key_version: "encryptionkey@0"
I20260812 07:12:58.804126 25039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:58.819621 25039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:58.823696 25039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:58.824014 25039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:58.833280 25039 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:46479
I20260812 07:12:58.833309 25125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:46479 every 8 connection(s)
I20260812 07:12:58.834438 25039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:58.837988 25126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:58.838596 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 25039
I20260812 07:12:58.838841 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:12:58.839272 32369 external_mini_cluster.cc:1428] Setting key a07336bbb14f0efd45a3d920cd0db942
I20260812 07:12:58.847064 25126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513: Bootstrap starting.
I20260812 07:12:58.850544 25126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513: Neither blocks nor log segments found. Creating new log.
Aug 12 07:12:58 dist-test-slave-csg5 krb5kdc[25024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:12:58.851521 25126 log.cc:824] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:58.853866 25126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513: No bootstrap required, opened a new log
I20260812 07:12:58.857319 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:58.841062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36967 (local address 127.31.156.126:46479)
0812 07:12:58.841572 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:58.841590 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:58.841630 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:12:58.842382 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:58.842402 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:58.842871 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:58.843220 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:58.844393 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:58.845553 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:58.846970 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:58.847754 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:58.847902 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:58.851542 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:58.851549 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:58.851565 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:58.851632 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:58.853740 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:58.854413 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:58.854419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:58.854424 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:58.854551 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:58.855017 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:58.855027 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:58.855030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:58.855518 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:58.855800 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:58.856196 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 07:12:58.856465 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":179,"threads_started":1}
I20260812 07:12:58.858286 25126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed3b1137b734d5191e8974261160513" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46479 } }
I20260812 07:12:58.858578 25126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:58.858685 25126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed3b1137b734d5191e8974261160513, State: Initialized, Role: FOLLOWER
I20260812 07:12:58.859176 25126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed3b1137b734d5191e8974261160513" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46479 } }
I20260812 07:12:58.859354 25126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:58.859440 25126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:58.859561 25126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:58.860878 25126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed3b1137b734d5191e8974261160513" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46479 } }
I20260812 07:12:58.860970 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:58.861344 25126 leader_election.cc:302] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ed3b1137b734d5191e8974261160513; no voters:
I20260812 07:12:58.861774 25126 leader_election.cc:288] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:58.861868 25132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:58.862138 25132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [term 1 LEADER]: Becoming Leader. State: Replica: 8ed3b1137b734d5191e8974261160513, State: Running, Role: LEADER
I20260812 07:12:58.862578 25132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed3b1137b734d5191e8974261160513" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46479 } }
I20260812 07:12:58.863466 25126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:58.864563 25134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ed3b1137b734d5191e8974261160513. Latest consensus state: current_term: 1 leader_uuid: "8ed3b1137b734d5191e8974261160513" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed3b1137b734d5191e8974261160513" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46479 } } }
I20260812 07:12:58.864773 25134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:58.865155 25133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ed3b1137b734d5191e8974261160513" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed3b1137b734d5191e8974261160513" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46479 } } }
I20260812 07:12:58.865253 25146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:58.865321 25133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:58.869678 25146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:58.875770 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:58.863523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60089 (local address 127.31.156.126:46479)
0812 07:12:58.863645 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:58.863660 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:58.866361 (+ 2701us) server_negotiation.cc:407] Connection header received
0812 07:12:58.866509 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:58.866512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:58.866568 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:58.866706 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:58.867468 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:58.868275 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:58.869533 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:58.869786 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:58.869898 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:58.871669 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:58.871673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:58.871676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:58.871708 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:58.873535 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:58.874169 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:58.874173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:58.874176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:58.874267 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:58.874733 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:58.874737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:58.874739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:58.874929 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:58.875045 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:58.875527 (+ 482us) server_negotiation.cc:299] Negotiation successful
0812 07:12:58.875641 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:12:58.877671 25146 catalog_manager.cc:1347] Generated new cluster ID: aa057074750843d9a985108981d92ad8
I20260812 07:12:58.877754 25146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:58.890411 25146 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:58.891944 25146 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:58.910091 25146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ed3b1137b734d5191e8974261160513: Generated new TSK 0
I20260812 07:12:58.911092 25146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:58.922739 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 25039
W20260812 07:12:58.933298 25022 connection.cc:535] client connection to 127.31.156.126:46479 recv error: Network error: failed to read from TLS socket (remote: 127.31.156.126:46479): Connection reset by peer (error 104)
2026-08-12T07:12:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1623 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:58 dist-test-slave-csg5 krb5kdc[25159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:58 dist-test-slave-csg5 krb5kdc[25159](info): set up 2 sockets
Aug 12 07:12:58 dist-test-slave-csg5 krb5kdc[25159](info): commencing operation
krb5kdc: starting...
W20260812 07:13:00.117873 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:13:00 dist-test-slave-csg5 krb5kdc[25159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:13:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:13:00.275314 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:37153
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:37153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:00.424063 25173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:00.424597 25173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:00.424714 25173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:00.435199 25173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:00.435336 25173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:00.435386 25173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:00.435426 25173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:00.435473 25173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:00.444260 25173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:37153
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:37153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:13:00.445657 25173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:00.447055 25173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:00.457383 25180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:00.457607 25173 server_base.cc:1034] running on GCE node
W20260812 07:13:00.457461 25179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:00.457383 25182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:00.458729 25173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:00.460223 25173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:00.461437 25173 hybrid_clock.cc:648] HybridClock initialized: now 1786518780461395 us; error 61 us; skew 500 ppm
Aug 12 07:13:00 dist-test-slave-csg5 krb5kdc[25159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518780, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:00.465131 25173 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:13:00.466475 25173 webserver.cc:460] Webserver started at http://127.31.156.126:35051/ using document root <none> and password file <none>
I20260812 07:13:00.467195 25173 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:00.467269 25173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:00.467525 25173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:00.470000 25173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "f641bb54a7b940db9e094de8287a4a2b"
format_stamp: "Formatted at 2026-08-12 07:13:00 on dist-test-slave-csg5"
server_key: "cc7ea555de603450827ee4a8d314348e"
server_key_iv: "1dcca813eda838cd469213505206a1c7"
server_key_version: "encryptionkey@0"
I20260812 07:13:00.470702 25173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "f641bb54a7b940db9e094de8287a4a2b"
format_stamp: "Formatted at 2026-08-12 07:13:00 on dist-test-slave-csg5"
server_key: "cc7ea555de603450827ee4a8d314348e"
server_key_iv: "1dcca813eda838cd469213505206a1c7"
server_key_version: "encryptionkey@0"
I20260812 07:13:00.475369 25173 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:13:00.478466 25191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:00.480228 25173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:13:00.480432 25173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "f641bb54a7b940db9e094de8287a4a2b"
format_stamp: "Formatted at 2026-08-12 07:13:00 on dist-test-slave-csg5"
server_key: "cc7ea555de603450827ee4a8d314348e"
server_key_iv: "1dcca813eda838cd469213505206a1c7"
server_key_version: "encryptionkey@0"
I20260812 07:13:00.480581 25173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:00.496219 25173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:00.500437 25173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:00.500694 25173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:00.509832 25173 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:37153
I20260812 07:13:00.509866 25245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:37153 every 8 connection(s)
I20260812 07:13:00.511009 25173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:13:00.514528 25246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:00.515583 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 25173
I20260812 07:13:00.515779 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:13:00.516106 32369 external_mini_cluster.cc:1428] Setting key e6548f7ff44a1e7aa854ce82f93e1ea4
I20260812 07:13:00.522555 25246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b: Bootstrap starting.
I20260812 07:13:00.525907 25246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:00.527047 25246 log.cc:824] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:00.528313 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:00.517792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54335 (local address 127.31.156.126:37153)
0812 07:13:00.518334 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:00.518356 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:00.518408 (+ 52us) server_negotiation.cc:407] Connection header received
0812 07:13:00.519380 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:00.519406 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:00.519899 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:00.520332 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:00.520836 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:00.522482 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:00.523688 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:00.524496 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:00.524711 (+ 215us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:00.525944 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:00.525952 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:00.525975 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:13:00.526586 (+ 611us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:00.526709 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:13:00.526750 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:00.526921 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:00.527145 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 07:13:00.527312 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":148,"threads_started":1}
I20260812 07:13:00.529729 25246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b: No bootstrap required, opened a new log
I20260812 07:13:00.532320 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:13:00.535308 25246 raft_consensus.cc:348] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f641bb54a7b940db9e094de8287a4a2b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37153 } }
I20260812 07:13:00.535640 25246 raft_consensus.cc:374] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:00.535712 25246 raft_consensus.cc:729] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f641bb54a7b940db9e094de8287a4a2b, State: Initialized, Role: FOLLOWER
I20260812 07:13:00.536257 25246 consensus_queue.cc:260] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f641bb54a7b940db9e094de8287a4a2b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37153 } }
I20260812 07:13:00.536418 25246 raft_consensus.cc:388] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:13:00.536522 25246 raft_consensus.cc:482] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:13:00.536659 25246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:00.537828 25246 raft_consensus.cc:504] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f641bb54a7b940db9e094de8287a4a2b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37153 } }
I20260812 07:13:00.538254 25246 leader_election.cc:302] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f641bb54a7b940db9e094de8287a4a2b; no voters:
I20260812 07:13:00.538767 25246 leader_election.cc:288] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:13:00.539209 25258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:13:00.539515 25258 raft_consensus.cc:686] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [term 1 LEADER]: Becoming Leader. State: Replica: f641bb54a7b940db9e094de8287a4a2b, State: Running, Role: LEADER
I20260812 07:13:00.540077 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:00.534807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39287 (local address 127.31.156.126:37153)
0812 07:13:00.534994 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:00.534999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:00.535119 (+ 120us) server_negotiation.cc:407] Connection header received
0812 07:13:00.535417 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:00.535421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:00.535482 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:00.535579 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:00.535887 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:00.536694 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:00.537886 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:00.538212 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:00.538336 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:00.539514 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:00.539519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:00.539528 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:13:00.539545 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:00.539579 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:13:00.539594 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:00.539674 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:00.539884 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 07:13:00.539922 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 07:13:00.540143 25258 consensus_queue.cc:237] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f641bb54a7b940db9e094de8287a4a2b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37153 } }
I20260812 07:13:00.540481 25246 sys_catalog.cc:564] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:00.542214 25259 sys_catalog.cc:455] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f641bb54a7b940db9e094de8287a4a2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f641bb54a7b940db9e094de8287a4a2b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37153 } } }
I20260812 07:13:00.542266 25260 sys_catalog.cc:455] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f641bb54a7b940db9e094de8287a4a2b. Latest consensus state: current_term: 1 leader_uuid: "f641bb54a7b940db9e094de8287a4a2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f641bb54a7b940db9e094de8287a4a2b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 37153 } } }
I20260812 07:13:00.542373 25259 sys_catalog.cc:458] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:00.542415 25260 sys_catalog.cc:458] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b [sys.catalog]: This master's current role is: LEADER
W20260812 07:13:00.545951 25278 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:13:00.546043 25278 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:13:00.546150 25275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:13:00.547106 25275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:13:00.554087 25275 catalog_manager.cc:1347] Generated new cluster ID: 2eacf9fd1d574e72bccc136cc0fc39ce
I20260812 07:13:00.554208 25275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:13:00.575556 25275 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:13:00.577044 25275 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:13:00.587530 25275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f641bb54a7b940db9e094de8287a4a2b: Generated new TSK 0
I20260812 07:13:00.588717 25275 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:13:00.596679 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 25173
W20260812 07:13:00.608759 25156 connection.cc:535] client connection to 127.31.156.126:37153 recv error: Network error: failed to read from TLS socket (remote: 127.31.156.126:37153): Connection reset by peer (error 104)
2026-08-12T07:13:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1675 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:13:00 dist-test-slave-csg5 krb5kdc[25285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:13:00 dist-test-slave-csg5 krb5kdc[25285](info): set up 2 sockets
Aug 12 07:13:00 dist-test-slave-csg5 krb5kdc[25285](info): commencing operation
krb5kdc: starting...
W20260812 07:13:01.800406 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:13:01 dist-test-slave-csg5 krb5kdc[25285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:13:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:13:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:13:01.960078 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46057
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:35901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:46057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:02.086488 25301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:02.086958 25301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:02.087055 25301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:02.091387 25301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:02.091509 25301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:02.091547 25301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:02.091589 25301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:02.091622 25301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:02.097932 25301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:35901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:46057
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:46057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:13:02.099254 25301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:02.100373 25301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:02.110280 25309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:02.110296 25307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:02.110570 25301 server_base.cc:1034] running on GCE node
W20260812 07:13:02.110283 25306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:02.111371 25301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:02.113009 25301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:02.114351 25301 hybrid_clock.cc:648] HybridClock initialized: now 1786518782114305 us; error 62 us; skew 500 ppm
Aug 12 07:13:02 dist-test-slave-csg5 krb5kdc[25285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518782, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:02.124642 25301 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:13:02.126039 25301 webserver.cc:460] Webserver started at http://127.31.156.126:44521/ using document root <none> and password file <none>
I20260812 07:13:02.126777 25301 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:02.126888 25301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:02.127162 25301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:02.129666 25301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "8e704c0fff53403f99c8bb52ba360575"
format_stamp: "Formatted at 2026-08-12 07:13:02 on dist-test-slave-csg5"
server_key: "5264f785831c1554c339b1b828ed946c"
server_key_iv: "48ebaa8a0b9b7b8905098a773e679e3e"
server_key_version: "encryptionkey@0"
I20260812 07:13:02.130357 25301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "8e704c0fff53403f99c8bb52ba360575"
format_stamp: "Formatted at 2026-08-12 07:13:02 on dist-test-slave-csg5"
server_key: "5264f785831c1554c339b1b828ed946c"
server_key_iv: "48ebaa8a0b9b7b8905098a773e679e3e"
server_key_version: "encryptionkey@0"
I20260812 07:13:02.134970 25301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:13:02.138144 25316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:02.139971 25301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:13:02.140195 25301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "8e704c0fff53403f99c8bb52ba360575"
format_stamp: "Formatted at 2026-08-12 07:13:02 on dist-test-slave-csg5"
server_key: "5264f785831c1554c339b1b828ed946c"
server_key_iv: "48ebaa8a0b9b7b8905098a773e679e3e"
server_key_version: "encryptionkey@0"
I20260812 07:13:02.140367 25301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:02.146302 25301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:02.150398 25301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:02.150636 25301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:02.158856 25301 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:46057
I20260812 07:13:02.158886 25373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:46057 every 8 connection(s)
I20260812 07:13:02.159955 25301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:13:02.163456 25374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:02.169345 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 25301
I20260812 07:13:02.169528 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:13:02.169849 32369 external_mini_cluster.cc:1428] Setting key 784eddafa9363f7ee9139b9202c7be46
I20260812 07:13:02.170825 25374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575: Bootstrap starting.
I20260812 07:13:02.173543 25374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:02.174655 25374 log.cc:824] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:02.176903 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:02.171651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53803 (local address 127.31.156.126:46057)
0812 07:13:02.172273 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:02.172289 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:02.172330 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:13:02.173121 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:02.173141 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:02.173607 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:02.174056 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:02.175126 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:02.175132 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:02.175146 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:13:02.175248 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:02.175338 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:13:02.175372 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:02.175506 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:02.175672 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 07:13:02.175851 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":172,"threads_started":1}
I20260812 07:13:02.177520 25374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575: No bootstrap required, opened a new log
I20260812 07:13:02.180424 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:13:02.181919 25374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e704c0fff53403f99c8bb52ba360575" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46057 } }
I20260812 07:13:02.182247 25374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:02.182333 25374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e704c0fff53403f99c8bb52ba360575, State: Initialized, Role: FOLLOWER
I20260812 07:13:02.182895 25374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e704c0fff53403f99c8bb52ba360575" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46057 } }
I20260812 07:13:02.183087 25374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:13:02.183178 25374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:13:02.183307 25374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:02.184544 25374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e704c0fff53403f99c8bb52ba360575" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46057 } }
I20260812 07:13:02.184962 25374 leader_election.cc:302] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e704c0fff53403f99c8bb52ba360575; no voters:
I20260812 07:13:02.185199 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:02.183082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48171 (local address 127.31.156.126:46057)
0812 07:13:02.183223 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:02.183229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:02.183500 (+ 271us) server_negotiation.cc:407] Connection header received
0812 07:13:02.183675 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:02.183678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:02.183719 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:02.183868 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:02.184723 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:02.184727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:02.184729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:13:02.184742 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:02.184778 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:13:02.184795 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:02.184896 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:02.184989 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 07:13:02.185038 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:13:02.185411 25374 leader_election.cc:288] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:13:02.185570 25387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:13:02.185774 25387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [term 1 LEADER]: Becoming Leader. State: Replica: 8e704c0fff53403f99c8bb52ba360575, State: Running, Role: LEADER
I20260812 07:13:02.186100 25387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e704c0fff53403f99c8bb52ba360575" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46057 } }
I20260812 07:13:02.186923 25374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:02.187479 25387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e704c0fff53403f99c8bb52ba360575. Latest consensus state: current_term: 1 leader_uuid: "8e704c0fff53403f99c8bb52ba360575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e704c0fff53403f99c8bb52ba360575" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46057 } } }
I20260812 07:13:02.187608 25387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:02.187529 25389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e704c0fff53403f99c8bb52ba360575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e704c0fff53403f99c8bb52ba360575" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 46057 } } }
I20260812 07:13:02.187834 25389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575 [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:02.188047 25392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:13:02.191881 25392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:13:02.199486 25392 catalog_manager.cc:1347] Generated new cluster ID: 8681102754344449943816e163940411
I20260812 07:13:02.199597 25392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:13:02.231571 25392 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:13:02.232049 25392 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:13:02.241146 25392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e704c0fff53403f99c8bb52ba360575: Generated new TSK 0
I20260812 07:13:02.242031 25392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:13:02.253327 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 25301
2026-08-12T07:13:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1656 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:13:02 dist-test-slave-csg5 krb5kdc[25409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:13:02 dist-test-slave-csg5 krb5kdc[25409](info): set up 2 sockets
Aug 12 07:13:02 dist-test-slave-csg5 krb5kdc[25409](info): commencing operation
krb5kdc: starting...
W20260812 07:13:03.469362 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:13:03 dist-test-slave-csg5 krb5kdc[25409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:13:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:13:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:13:03.631285 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:34381
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:34381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:03.757305 25426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:03.757635 25426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:03.757751 25426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:03.762126 25426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:03.762223 25426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:03.762243 25426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:03.762260 25426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:03.762276 25426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:03.768514 25426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:34381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:34381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:13:03.769801 25426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:03.770963 25426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:03.780438 25435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:03.780503 25426 server_base.cc:1034] running on GCE node
W20260812 07:13:03.780438 25432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:03.780441 25433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:03.781472 25426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:03.782831 25426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:03.784112 25426 hybrid_clock.cc:648] HybridClock initialized: now 1786518783784075 us; error 45 us; skew 500 ppm
Aug 12 07:13:03 dist-test-slave-csg5 krb5kdc[25409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518783, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:03.787508 25426 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:13:03.788589 25426 webserver.cc:460] Webserver started at http://127.31.156.126:38471/ using document root <none> and password file <none>
I20260812 07:13:03.789211 25426 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:03.789310 25426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:03.789562 25426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:03.791904 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "6d587de47389444eb13f490b2de16fc0"
format_stamp: "Formatted at 2026-08-12 07:13:03 on dist-test-slave-csg5"
server_key: "81828d415e169494dc14fae577d6dfb4"
server_key_iv: "ecee6e1b0c02f9bc052ae6c64aa48052"
server_key_version: "encryptionkey@0"
I20260812 07:13:03.792537 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "6d587de47389444eb13f490b2de16fc0"
format_stamp: "Formatted at 2026-08-12 07:13:03 on dist-test-slave-csg5"
server_key: "81828d415e169494dc14fae577d6dfb4"
server_key_iv: "ecee6e1b0c02f9bc052ae6c64aa48052"
server_key_version: "encryptionkey@0"
I20260812 07:13:03.796919 25426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:13:03.800341 25445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:03.801805 25426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260812 07:13:03.802006 25426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "6d587de47389444eb13f490b2de16fc0"
format_stamp: "Formatted at 2026-08-12 07:13:03 on dist-test-slave-csg5"
server_key: "81828d415e169494dc14fae577d6dfb4"
server_key_iv: "ecee6e1b0c02f9bc052ae6c64aa48052"
server_key_version: "encryptionkey@0"
I20260812 07:13:03.802150 25426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:03.819883 25426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:03.823722 25426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:03.823976 25426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:03.832010 25426 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:34381
I20260812 07:13:03.832069 25501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:34381 every 8 connection(s)
I20260812 07:13:03.833101 25426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:13:03.836519 25502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:03.840855 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 25426
I20260812 07:13:03.841001 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:13:03.841305 32369 external_mini_cluster.cc:1428] Setting key aba8a76b743cbebef63ed0cf5dfcf59e
I20260812 07:13:03.843585 25502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0: Bootstrap starting.
I20260812 07:13:03.846760 25502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:03.847707 25502 log.cc:824] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:03.850333 25502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0: No bootstrap required, opened a new log
Aug 12 07:13:03 dist-test-slave-csg5 krb5kdc[25409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:13:03.854987 25502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d587de47389444eb13f490b2de16fc0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34381 } }
I20260812 07:13:03.855321 25502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:03.855417 25502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d587de47389444eb13f490b2de16fc0, State: Initialized, Role: FOLLOWER
I20260812 07:13:03.856029 25502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d587de47389444eb13f490b2de16fc0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34381 } }
I20260812 07:13:03.856268 25502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:13:03.856355 25502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:13:03.856472 25502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:03.857920 25502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d587de47389444eb13f490b2de16fc0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34381 } }
I20260812 07:13:03.858491 25502 leader_election.cc:302] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d587de47389444eb13f490b2de16fc0; no voters:
I20260812 07:13:03.858997 25502 leader_election.cc:288] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:13:03.859285 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:03.842944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60751 (local address 127.31.156.126:34381)
0812 07:13:03.843436 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:03.843458 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:03.843507 (+ 49us) server_negotiation.cc:407] Connection header received
0812 07:13:03.844457 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:03.844483 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:03.844997 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:03.845471 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:03.847068 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:03.848451 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:03.849433 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:03.849943 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:03.850127 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:03.852864 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:03.852870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:03.852887 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:03.852921 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:03.854964 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:03.855546 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:03.855554 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:03.855561 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:03.855736 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:03.856143 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:03.856147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:03.856149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:03.856828 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:03.857078 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:03.857475 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 07:13:03.857891 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":118,"threads_started":1}
I20260812 07:13:03.859338 25507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:13:03.860098 25507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [term 1 LEADER]: Becoming Leader. State: Replica: 6d587de47389444eb13f490b2de16fc0, State: Running, Role: LEADER
I20260812 07:13:03.860527 25507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d587de47389444eb13f490b2de16fc0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34381 } }
I20260812 07:13:03.861183 25502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:03.862776 25508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d587de47389444eb13f490b2de16fc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d587de47389444eb13f490b2de16fc0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34381 } } }
I20260812 07:13:03.862941 25508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:03.863379 25509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d587de47389444eb13f490b2de16fc0. Latest consensus state: current_term: 1 leader_uuid: "6d587de47389444eb13f490b2de16fc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d587de47389444eb13f490b2de16fc0" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34381 } } }
I20260812 07:13:03.863551 25509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:03.864143 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 07:13:03.867954 25523 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:13:03.868090 25523 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:13:03.868165 25517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:13:03.869150 25517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:13:03.877094 25517 catalog_manager.cc:1347] Generated new cluster ID: 00f657b7611e4b9cb49369cf2c2900e4
I20260812 07:13:03.877192 25517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:13:03.877489 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:03.867410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39543 (local address 127.31.156.126:34381)
0812 07:13:03.867549 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:03.867554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:03.867570 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:13:03.867614 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:03.867617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:03.867667 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:03.867760 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:03.869130 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:03.869760 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:03.870749 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:03.871032 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:03.871159 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:03.873419 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:03.873424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:03.873427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:03.873460 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:03.875310 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:03.875833 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:03.875837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:03.875839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:03.875938 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:03.876280 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:03.876284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:03.876286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:03.876538 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:03.876675 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:03.877149 (+ 474us) server_negotiation.cc:299] Negotiation successful
0812 07:13:03.877305 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:13:03.884917 25517 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:13:03.886319 25517 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:13:03.896860 25517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d587de47389444eb13f490b2de16fc0: Generated new TSK 0
I20260812 07:13:03.897892 25517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:13:03.905130 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 25426
2026-08-12T07:13:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1650 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:13:03 dist-test-slave-csg5 krb5kdc[25534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:13:03 dist-test-slave-csg5 krb5kdc[25534](info): set up 2 sockets
Aug 12 07:13:03 dist-test-slave-csg5 krb5kdc[25534](info): commencing operation
krb5kdc: starting...
W20260812 07:13:05.152654 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:13:05 dist-test-slave-csg5 krb5kdc[25534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:13:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:13:05.319914 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:36127
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:34823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:36127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:05.448663 25551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:05.449090 25551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:05.449203 25551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:05.453730 25551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:05.453874 25551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:05.453912 25551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:05.453954 25551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:05.453991 25551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:05.460479 25551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:34823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:36127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:36127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:13:05.461812 25551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:05.462980 25551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:05.473857 25557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:05.473850 25559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:05.473850 25556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:05.474112 25551 server_base.cc:1034] running on GCE node
I20260812 07:13:05.475049 25551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:05.476366 25551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:05.477572 25551 hybrid_clock.cc:648] HybridClock initialized: now 1786518785477549 us; error 38 us; skew 500 ppm
Aug 12 07:13:05 dist-test-slave-csg5 krb5kdc[25534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518785, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:05.481045 25551 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:13:05.482288 25551 webserver.cc:460] Webserver started at http://127.31.156.126:40585/ using document root <none> and password file <none>
I20260812 07:13:05.482985 25551 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:05.483059 25551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:05.483254 25551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:05.485581 25551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "221257f2093c4a8e82cee1deb6f72035"
format_stamp: "Formatted at 2026-08-12 07:13:05 on dist-test-slave-csg5"
server_key: "6b32f4fc026199824f9e9090b74853c5"
server_key_iv: "d21121521ba7d73435471d17048e669d"
server_key_version: "encryptionkey@0"
I20260812 07:13:05.486249 25551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "221257f2093c4a8e82cee1deb6f72035"
format_stamp: "Formatted at 2026-08-12 07:13:05 on dist-test-slave-csg5"
server_key: "6b32f4fc026199824f9e9090b74853c5"
server_key_iv: "d21121521ba7d73435471d17048e669d"
server_key_version: "encryptionkey@0"
I20260812 07:13:05.490888 25551 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:13:05.494902 25567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:05.496755 25551 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:13:05.496985 25551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "221257f2093c4a8e82cee1deb6f72035"
format_stamp: "Formatted at 2026-08-12 07:13:05 on dist-test-slave-csg5"
server_key: "6b32f4fc026199824f9e9090b74853c5"
server_key_iv: "d21121521ba7d73435471d17048e669d"
server_key_version: "encryptionkey@0"
I20260812 07:13:05.497148 25551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:05.513705 25551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:05.517652 25551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:05.517926 25551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:05.526317 25551 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:36127
I20260812 07:13:05.526472 25626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:36127 every 8 connection(s)
I20260812 07:13:05.527501 25551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:13:05.529842 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 25551
I20260812 07:13:05.530010 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:13:05.530318 32369 external_mini_cluster.cc:1428] Setting key 4118ded6284bb3a865b4baba9d6279ef
I20260812 07:13:05.530959 25627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:05.538781 25627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035: Bootstrap starting.
I20260812 07:13:05.540699 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:05.531963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59109 (local address 127.31.156.126:36127)
0812 07:13:05.532463 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:05.532481 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:05.532521 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:13:05.533222 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:05.533242 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:05.533606 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:05.533969 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:05.534439 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:05.535526 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:05.536851 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:05.537396 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:05.537542 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:05.538568 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:05.538574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:05.538589 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:13:05.539227 (+ 638us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:05.539295 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:13:05.539321 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:05.539407 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:05.539584 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 07:13:05.539894 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":167,"threads_started":1}
I20260812 07:13:05.541396 25627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:05.542331 25627 log.cc:824] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:05.544108 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:13:05.544970 25627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035: No bootstrap required, opened a new log
I20260812 07:13:05.549501 25627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221257f2093c4a8e82cee1deb6f72035" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36127 } }
I20260812 07:13:05.549845 25627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:05.549919 25627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 221257f2093c4a8e82cee1deb6f72035, State: Initialized, Role: FOLLOWER
I20260812 07:13:05.550472 25627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221257f2093c4a8e82cee1deb6f72035" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36127 } }
I20260812 07:13:05.550649 25627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:13:05.550750 25627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:13:05.550880 25627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:05.551839 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:05.546500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37203 (local address 127.31.156.126:36127)
0812 07:13:05.546647 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:05.546690 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:05.546892 (+ 202us) server_negotiation.cc:407] Connection header received
0812 07:13:05.546982 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:05.546987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:05.547040 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:05.547142 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:05.547397 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:05.548884 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:05.549824 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:05.550165 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:05.550284 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:05.551266 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:05.551270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:05.551273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:13:05.551288 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:05.551319 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:13:05.551333 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:05.551416 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:05.551622 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 07:13:05.551689 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:13:05.552227 25627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221257f2093c4a8e82cee1deb6f72035" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36127 } }
I20260812 07:13:05.552623 25627 leader_election.cc:302] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 221257f2093c4a8e82cee1deb6f72035; no voters:
I20260812 07:13:05.553011 25627 leader_election.cc:288] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:13:05.553099 25641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:13:05.553416 25641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [term 1 LEADER]: Becoming Leader. State: Replica: 221257f2093c4a8e82cee1deb6f72035, State: Running, Role: LEADER
I20260812 07:13:05.553810 25641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221257f2093c4a8e82cee1deb6f72035" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36127 } }
I20260812 07:13:05.554587 25627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:05.555764 25642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 221257f2093c4a8e82cee1deb6f72035. Latest consensus state: current_term: 1 leader_uuid: "221257f2093c4a8e82cee1deb6f72035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221257f2093c4a8e82cee1deb6f72035" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36127 } } }
I20260812 07:13:05.555917 25642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:05.556454 25652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:13:05.557143 25643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "221257f2093c4a8e82cee1deb6f72035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221257f2093c4a8e82cee1deb6f72035" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 36127 } } }
I20260812 07:13:05.557366 25643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035 [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:05.560144 25652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:13:05.567018 25652 catalog_manager.cc:1347] Generated new cluster ID: 4c2184cfd09e4d7f8e3fd880223413a8
I20260812 07:13:05.567123 25652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:13:05.583644 25652 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:13:05.585021 25652 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:13:05.595656 25652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 221257f2093c4a8e82cee1deb6f72035: Generated new TSK 0
I20260812 07:13:05.596714 25652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:13:05.613114 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 25551
2026-08-12T07:13:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1707 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:13:05 dist-test-slave-csg5 krb5kdc[25664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:13:05 dist-test-slave-csg5 krb5kdc[25664](info): set up 2 sockets
Aug 12 07:13:05 dist-test-slave-csg5 krb5kdc[25664](info): commencing operation
krb5kdc: starting...
W20260812 07:13:06.775264 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:13:06 dist-test-slave-csg5 krb5kdc[25664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:13:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:13:06.935633 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:42975
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:42597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:42975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:07.066071 25679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:07.066409 25679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:07.066457 25679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:07.070950 25679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:07.071048 25679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:07.071069 25679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:07.071086 25679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:07.071101 25679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:07.077142 25679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:42597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:42975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:42975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:13:07.078440 25679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:07.079627 25679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:07.090982 25684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:07.091053 25679 server_base.cc:1034] running on GCE node
W20260812 07:13:07.090874 25685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:07.090993 25687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:07.091893 25679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:07.093232 25679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:07.094463 25679 hybrid_clock.cc:648] HybridClock initialized: now 1786518787094432 us; error 47 us; skew 500 ppm
Aug 12 07:13:07 dist-test-slave-csg5 krb5kdc[25664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518787, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:07.097718 25679 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:13:07.099102 25679 webserver.cc:460] Webserver started at http://127.31.156.126:39961/ using document root <none> and password file <none>
I20260812 07:13:07.099763 25679 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:07.099874 25679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:07.100137 25679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:07.102474 25679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "6de2142919e04c118b09c666c9e21739"
format_stamp: "Formatted at 2026-08-12 07:13:07 on dist-test-slave-csg5"
server_key: "434f9d29025fcf5bfa705227964238af"
server_key_iv: "f3c22ee78fce280f1541025389b01e2c"
server_key_version: "encryptionkey@0"
I20260812 07:13:07.103211 25679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "6de2142919e04c118b09c666c9e21739"
format_stamp: "Formatted at 2026-08-12 07:13:07 on dist-test-slave-csg5"
server_key: "434f9d29025fcf5bfa705227964238af"
server_key_iv: "f3c22ee78fce280f1541025389b01e2c"
server_key_version: "encryptionkey@0"
I20260812 07:13:07.108325 25679 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:13:07.111238 25694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:07.112879 25679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:13:07.113075 25679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "6de2142919e04c118b09c666c9e21739"
format_stamp: "Formatted at 2026-08-12 07:13:07 on dist-test-slave-csg5"
server_key: "434f9d29025fcf5bfa705227964238af"
server_key_iv: "f3c22ee78fce280f1541025389b01e2c"
server_key_version: "encryptionkey@0"
I20260812 07:13:07.113225 25679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:07.118544 25679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:07.122411 25679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:07.122700 25679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:07.131099 25679 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:42975
I20260812 07:13:07.131153 25758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:42975 every 8 connection(s)
I20260812 07:13:07.132243 25679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:13:07.135605 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 25679
I20260812 07:13:07.135727 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:13:07.135711 25759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:07.136037 32369 external_mini_cluster.cc:1428] Setting key 6965b7032875e571d05a780dbc681285
I20260812 07:13:07.143828 25759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739: Bootstrap starting.
I20260812 07:13:07.146752 25759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:07.148327 25759 log.cc:824] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:07.149718 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:07.137601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33533 (local address 127.31.156.126:42975)
0812 07:13:07.138104 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:07.138263 (+ 159us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:07.138321 (+ 58us) server_negotiation.cc:407] Connection header received
0812 07:13:07.139346 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:07.139368 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:07.139682 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:07.139994 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:07.146544 (+ 6550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:07.146551 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:07.146566 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:13:07.146582 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:07.146956 (+ 374us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:13:07.147135 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:07.147316 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:07.148327 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0812 07:13:07.148592 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":195,"threads_started":1}
I20260812 07:13:07.151080 25759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739: No bootstrap required, opened a new log
I20260812 07:13:07.153291 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:13:07.155438 25759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de2142919e04c118b09c666c9e21739" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42975 } }
I20260812 07:13:07.155804 25759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:07.155884 25759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de2142919e04c118b09c666c9e21739, State: Initialized, Role: FOLLOWER
I20260812 07:13:07.156436 25759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de2142919e04c118b09c666c9e21739" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42975 } }
I20260812 07:13:07.156612 25759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:13:07.156708 25759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:13:07.156841 25759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:07.157691 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:07.155757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37941 (local address 127.31.156.126:42975)
0812 07:13:07.155904 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:07.155909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:07.156161 (+ 252us) server_negotiation.cc:407] Connection header received
0812 07:13:07.156326 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:07.156329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:07.156377 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:07.156466 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:07.157256 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:07.157260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:07.157262 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:13:07.157274 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:07.157309 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:13:07.157322 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:07.157421 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:07.157500 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 07:13:07.157549 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:13:07.158113 25759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de2142919e04c118b09c666c9e21739" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42975 } }
I20260812 07:13:07.158511 25759 leader_election.cc:302] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6de2142919e04c118b09c666c9e21739; no voters:
I20260812 07:13:07.158948 25759 leader_election.cc:288] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:13:07.159253 25769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:13:07.159507 25769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [term 1 LEADER]: Becoming Leader. State: Replica: 6de2142919e04c118b09c666c9e21739, State: Running, Role: LEADER
I20260812 07:13:07.159869 25769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de2142919e04c118b09c666c9e21739" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42975 } }
I20260812 07:13:07.160338 25759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:07.161607 25771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6de2142919e04c118b09c666c9e21739. Latest consensus state: current_term: 1 leader_uuid: "6de2142919e04c118b09c666c9e21739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de2142919e04c118b09c666c9e21739" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42975 } } }
I20260812 07:13:07.161756 25771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:07.162199 25778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:13:07.162454 25770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6de2142919e04c118b09c666c9e21739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de2142919e04c118b09c666c9e21739" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 42975 } } }
I20260812 07:13:07.162575 25770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739 [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:07.166007 25778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:13:07.173756 25778 catalog_manager.cc:1347] Generated new cluster ID: 2bc125783f28462b9b3607fe67ba5208
I20260812 07:13:07.173889 25778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:13:07.185936 25778 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:13:07.186362 25778 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:13:07.201705 25778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6de2142919e04c118b09c666c9e21739: Generated new TSK 0
I20260812 07:13:07.202867 25778 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:13:07.213482 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 25679
2026-08-12T07:13:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1602 ms)
[----------] 18 tests from AuthTokenIssuingTest (29677 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:13:07 dist-test-slave-csg5 krb5kdc[25793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:13:07 dist-test-slave-csg5 krb5kdc[25793](info): set up 2 sockets
Aug 12 07:13:07 dist-test-slave-csg5 krb5kdc[25793](info): commencing operation
krb5kdc: starting...
W20260812 07:13:08.436342 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:13:08 dist-test-slave-csg5 krb5kdc[25793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:13:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:13:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:13:08.595010 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:35863
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:35863,127.31.156.125:46199,127.31.156.124:34245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:08.724967 25808 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:13:08.725425 25808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:08.725494 25808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:08.725572 25808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:08.730077 25808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:08.730212 25808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:08.730252 25808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:08.730294 25808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:08.730330 25808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:08.736209 25808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:35863,127.31.156.125:46199,127.31.156.124:34245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:35863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:13:08.737583 25808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:08.738842 25808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:08.749095 25813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:08.749089 25816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:08.749477 25814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:08.749522 25808 server_base.cc:1034] running on GCE node
I20260812 07:13:08.750265 25808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:08.751631 25808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:08.752851 25808 hybrid_clock.cc:648] HybridClock initialized: now 1786518788752823 us; error 47 us; skew 500 ppm
Aug 12 07:13:08 dist-test-slave-csg5 krb5kdc[25793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518788, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:08.756224 25808 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:13:08.757411 25808 webserver.cc:460] Webserver started at http://127.31.156.126:39639/ using document root <none> and password file <none>
I20260812 07:13:08.758087 25808 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:08.758203 25808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:08.758484 25808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:08.761072 25808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "9f63d551b072455ba653869528cff633"
format_stamp: "Formatted at 2026-08-12 07:13:08 on dist-test-slave-csg5"
server_key: "44e2b1238f8db710ad7c0f5cfebcf197"
server_key_iv: "3f1c672a305b963a18de4d7a4c1bd8c6"
server_key_version: "encryptionkey@0"
I20260812 07:13:08.761772 25808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "9f63d551b072455ba653869528cff633"
format_stamp: "Formatted at 2026-08-12 07:13:08 on dist-test-slave-csg5"
server_key: "44e2b1238f8db710ad7c0f5cfebcf197"
server_key_iv: "3f1c672a305b963a18de4d7a4c1bd8c6"
server_key_version: "encryptionkey@0"
I20260812 07:13:08.766588 25808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:13:08.769625 25824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:08.771265 25808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:13:08.771436 25808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "9f63d551b072455ba653869528cff633"
format_stamp: "Formatted at 2026-08-12 07:13:08 on dist-test-slave-csg5"
server_key: "44e2b1238f8db710ad7c0f5cfebcf197"
server_key_iv: "3f1c672a305b963a18de4d7a4c1bd8c6"
server_key_version: "encryptionkey@0"
I20260812 07:13:08.771579 25808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:08.795185 25808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:08.799469 25808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:08.799728 25808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:08.809499 25808 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:35863
I20260812 07:13:08.809896 25882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:35863 every 8 connection(s)
I20260812 07:13:08.810843 25808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:13:08.814116 25883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:08.814435 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 25808
I20260812 07:13:08.814579 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:13:08.814925 32369 external_mini_cluster.cc:1428] Setting key 6ec89b09a5a79d3a87562576d496dbbd
I20260812 07:13:08.819607 25883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.156.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.125@KRBTEST.COM" created.
Aug 12 07:13:08 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518788, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:13:08.838142 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:08.822964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:35863 (local address 127.0.0.1:40480)
0812 07:13:08.823650 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:08.823693 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:08.824655 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:08.825421 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:08.825427 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:08.825915 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:08.826188 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:08.826204 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:08.827593 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:08.827596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:08.828357 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:08.828367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:08.829283 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:08.829295 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:08.829406 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:08.830107 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:08.830126 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:08.832141 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:08.834701 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:08.834712 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:08.834727 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:08.834980 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:08.835420 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:08.835423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:08.835424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:08.835543 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:08.836273 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:08.836280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:08.836426 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 07:13:08.836605 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 07:13:08.837025 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":118,"spinlock_wait_cycles":17152,"thread_start_us":220,"threads_started":1}
I20260812 07:13:08.838142 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:08.822964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40480 (local address 127.31.156.126:35863)
0812 07:13:08.823489 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:08.823516 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:08.823820 (+ 304us) server_negotiation.cc:407] Connection header received
0812 07:13:08.824996 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:08.825001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:08.825214 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:08.825422 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:08.826349 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:08.827416 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:08.828500 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:08.828915 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:08.829122 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:08.832342 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:08.832349 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:08.832362 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:08.832386 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:08.834532 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:08.835110 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:08.835118 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:08.835126 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:08.835299 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:08.835650 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:08.835653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:08.835655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:08.836107 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:08.836289 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:08.836685 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 07:13:08.836968 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":176,"threads_started":1}
I20260812 07:13:08.841173 25883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:13:08.842043 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:08.841736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46199 (local address 127.0.0.1:34554)
0812 07:13:08.841881 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:08.841964 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 07:13:08.842432 25827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
I20260812 07:13:08.843532 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:08.843267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46199 (local address 127.0.0.1:34566)
0812 07:13:08.843384 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:08.843459 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 07:13:08.843770 25883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:46199: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab.
Entry for principal kudu/127.31.156.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab.
I20260812 07:13:08.851584 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.125:46199
--webserver_interface=127.31.156.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab
--principal=kudu/127.31.156.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:35863,127.31.156.125:46199,127.31.156.124:34245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:13:08.893069 25883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } attempt: 1
I20260812 07:13:08.894189 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:08.893845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46199 (local address 127.0.0.1:34568)
0812 07:13:08.894029 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:08.894098 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 07:13:08.895193 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:08.894912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46199 (local address 127.0.0.1:34572)
0812 07:13:08.895045 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:08.895111 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 07:13:08.895413 25883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:46199: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
I20260812 07:13:08.963635 25883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } attempt: 2
I20260812 07:13:08.964751 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:08.964338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46199 (local address 127.0.0.1:34584)
0812 07:13:08.964548 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:08.964634 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 07:13:08.965721 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:08.965432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46199 (local address 127.0.0.1:34586)
0812 07:13:08.965573 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:08.965642 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 07:13:08.965929 25883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:46199: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
W20260812 07:13:08.983899 25891 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:13:08.984356 25891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:08.984428 25891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:08.984519 25891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:08.989611 25891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:08.989722 25891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:08.989760 25891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:08.989809 25891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:08.989857 25891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:08.997000 25891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:35863,127.31.156.125:46199,127.31.156.124:34245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab
--principal=kudu/127.31.156.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.125:46199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.156.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:13:08.998373 25891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:08.999650 25891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:13:09.009552 25891 server_base.cc:1034] running on GCE node
W20260812 07:13:09.009454 25898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:09.009462 25896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:09.009819 25900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:09.010596 25891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:09.012023 25891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:09.013326 25891 hybrid_clock.cc:648] HybridClock initialized: now 1786518789013248 us; error 43 us; skew 500 ppm
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518789, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:09.017521 25891 init.cc:377] Logged in from keytab as kudu/127.31.156.125@KRBTEST.COM (short username kudu)
I20260812 07:13:09.019011 25891 webserver.cc:460] Webserver started at http://127.31.156.125:42547/ using document root <none> and password file <none>
I20260812 07:13:09.019667 25891 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:09.019794 25891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:09.020025 25891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:09.022339 25891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/data/instance:
uuid: "0d89e923ef9a46e4857af6998ff4b634"
format_stamp: "Formatted at 2026-08-12 07:13:09 on dist-test-slave-csg5"
server_key: "76378f266d86199b897457d9d8bba533"
server_key_iv: "4aafea07bcba6da4745c2c101f5b62c8"
server_key_version: "encryptionkey@0"
I20260812 07:13:09.023059 25891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/wal/instance:
uuid: "0d89e923ef9a46e4857af6998ff4b634"
format_stamp: "Formatted at 2026-08-12 07:13:09 on dist-test-slave-csg5"
server_key: "76378f266d86199b897457d9d8bba533"
server_key_iv: "4aafea07bcba6da4745c2c101f5b62c8"
server_key_version: "encryptionkey@0"
I20260812 07:13:09.027603 25891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:13:09.030970 25909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:09.032442 25891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:13:09.032606 25891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/wal
uuid: "0d89e923ef9a46e4857af6998ff4b634"
format_stamp: "Formatted at 2026-08-12 07:13:09 on dist-test-slave-csg5"
server_key: "76378f266d86199b897457d9d8bba533"
server_key_iv: "4aafea07bcba6da4745c2c101f5b62c8"
server_key_version: "encryptionkey@0"
I20260812 07:13:09.032744 25891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:09.054080 25891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:09.057235 25883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } attempt: 3
I20260812 07:13:09.057804 25891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:09.058030 25891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:09.058252 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.057861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46199 (local address 127.0.0.1:34598)
0812 07:13:09.058033 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.058119 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 07:13:09.059202 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.058904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46199 (local address 127.0.0.1:34602)
0812 07:13:09.059047 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.059120 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 07:13:09.059429 25883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:46199: Network error: Client connection negotiation failed: client connection to 127.31.156.125:46199: connect: Connection refused (error 111)
I20260812 07:13:09.066820 25891 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.125:46199
I20260812 07:13:09.067234 25968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.125:46199 every 8 connection(s)
I20260812 07:13:09.068001 25891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
I20260812 07:13:09.071002 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 25891
I20260812 07:13:09.071117 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-1/wal/instance
I20260812 07:13:09.071538 32369 external_mini_cluster.cc:1428] Setting key 5c1da50c47ac33b1a35e7df3f2918f19
I20260812 07:13:09.071859 25969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:09.076843 25969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.156.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.124@KRBTEST.COM" created.
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518789, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:13:09.094329 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.079198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40490 (local address 127.31.156.126:35863)
0812 07:13:09.079361 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.079365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.080782 (+ 1417us) server_negotiation.cc:407] Connection header received
0812 07:13:09.081918 (+ 1136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.081922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.081994 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.082107 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.083561 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.084139 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.085486 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.085833 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.085944 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.088918 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.088924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.088931 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.088962 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.091470 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.092393 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.092397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.092399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.092503 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.092935 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.092938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.092940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.093184 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.093316 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.093972 (+ 656us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.094125 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:13:09.095325 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.079840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:35863 (local address 127.0.0.1:40490)
0812 07:13:09.080549 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.080613 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:09.081653 (+ 1040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:09.082312 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:09.082320 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:09.082765 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:09.083389 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.083405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.084306 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.084309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.085319 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.085333 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.086047 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.086051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.086231 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.086998 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:09.087018 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:09.088682 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:09.091640 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.091649 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.091672 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.092189 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.092611 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.092615 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.092617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.092818 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.093352 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:09.093361 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:09.093675 (+ 314us) client_negotiation.cc:769] Sending connection context
0812 07:13:09.093905 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 07:13:09.094292 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":196,"threads_started":1}
I20260812 07:13:09.096915 25969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518789, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab.
Entry for principal kudu/127.31.156.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab.
I20260812 07:13:09.108057 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.124:34245
--webserver_interface=127.31.156.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab
--principal=kudu/127.31.156.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:35863,127.31.156.125:46199,127.31.156.124:34245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:13:09.108111 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.097525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46199 (local address 127.0.0.1:34610)
0812 07:13:09.097632 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.097648 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:09.097737 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:09.098490 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:09.098493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:09.098747 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:09.098843 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.098851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.099845 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.099851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.100380 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.100388 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.101025 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.101029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.101118 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.101826 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:09.101839 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:09.103638 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:09.105994 (+ 2356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.106000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.106003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.106212 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.106583 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.106586 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.106588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.106803 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.107457 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:09.107461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:09.107596 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 07:13:09.107738 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 07:13:09.107921 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 07:13:09.108197 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.097698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.31.156.125:46199)
0812 07:13:09.098089 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.098098 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.098118 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:13:09.098156 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.098160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.098347 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.098517 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.098966 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.099713 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.100500 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.100900 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.101021 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.103848 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.103860 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.103876 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.103903 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.105862 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.106348 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.106353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.106355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.106463 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.106919 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.106925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.106929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.107320 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.107449 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.107888 (+ 439us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.108029 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":179,"threads_started":1}
I20260812 07:13:09.110745 25969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:13:09.111709 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.111362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60250)
0812 07:13:09.111533 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.111624 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260812 07:13:09.111975 25913 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
I20260812 07:13:09.113173 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.112901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60252)
0812 07:13:09.113038 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.113101 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 07:13:09.113583 25969 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:34245: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
I20260812 07:13:09.162969 25969 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } attempt: 1
I20260812 07:13:09.164284 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.163756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60258)
0812 07:13:09.163989 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.164075 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260812 07:13:09.165477 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.165135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60260)
0812 07:13:09.165335 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.165400 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260812 07:13:09.165786 25969 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:34245: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
I20260812 07:13:09.202637 25883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } attempt: 4
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518788, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
I20260812 07:13:09.214855 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.203528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46199 (local address 127.0.0.1:34622)
0812 07:13:09.203769 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.203795 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:09.203899 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:09.204289 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:09.204292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:09.204497 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:09.204613 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.204620 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.205866 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.205870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.206484 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.206494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.207234 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.207237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.207338 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.207985 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:09.207998 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:09.209810 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:09.212529 (+ 2719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.212532 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.212534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.212787 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.213385 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.213388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.213390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.213497 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.214147 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:09.214150 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:09.214338 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 07:13:09.214442 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 07:13:09.214590 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260812 07:13:09.214994 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.203688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34622 (local address 127.31.156.125:46199)
0812 07:13:09.203844 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.203849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.203871 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:13:09.203998 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.204001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.204060 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.204153 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.204772 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.205636 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.206616 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.207051 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.207211 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.210017 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.210024 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.210031 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.210073 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.212319 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.212948 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.212954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.212956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.213081 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.213664 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.213673 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.213675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.214004 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.214148 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.214565 (+ 417us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.214794 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:13:09.215932 25883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:13:09.216945 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.216646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60270)
0812 07:13:09.216809 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.216872 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 07:13:09.217954 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.217674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60276)
0812 07:13:09.217833 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.217888 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260812 07:13:09.218207 25883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:34245: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
I20260812 07:13:09.234170 25969 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } attempt: 2
I20260812 07:13:09.235322 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.234974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60284)
0812 07:13:09.235148 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.235230 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 07:13:09.236382 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.236022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60288)
0812 07:13:09.236192 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.236265 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260812 07:13:09.236660 25969 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:34245: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
W20260812 07:13:09.253834 25976 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:13:09.254210 25976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:09.254240 25976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:09.254295 25976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:09.259632 25976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:09.259730 25976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:09.259759 25976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:09.259781 25976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:09.259799 25976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:09.266904 25976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:35863,127.31.156.125:46199,127.31.156.124:34245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab
--principal=kudu/127.31.156.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.124:34245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.156.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:13:09.268517 25976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:09.269909 25976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:13:09.277395 25883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } attempt: 1
I20260812 07:13:09.278461 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.278033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60290)
0812 07:13:09.278275 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.278347 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260812 07:13:09.279965 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.279561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60298)
0812 07:13:09.279825 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.279880 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260812 07:13:09.280280 25982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:09.280350 25985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:09.280486 25983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:09.280637 25883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:34245: Network error: Client connection negotiation failed: client connection to 127.31.156.124:34245: connect: Connection refused (error 111)
I20260812 07:13:09.280949 25976 server_base.cc:1034] running on GCE node
I20260812 07:13:09.281664 25976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:09.283103 25976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:09.284333 25976 hybrid_clock.cc:648] HybridClock initialized: now 1786518789284309 us; error 38 us; skew 500 ppm
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518789, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:09.288443 25976 init.cc:377] Logged in from keytab as kudu/127.31.156.124@KRBTEST.COM (short username kudu)
I20260812 07:13:09.289683 25976 webserver.cc:460] Webserver started at http://127.31.156.124:34091/ using document root <none> and password file <none>
I20260812 07:13:09.290356 25976 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:09.290467 25976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:09.290843 25976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:09.293480 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/data/instance:
uuid: "b966954119674488a62ab18248303617"
format_stamp: "Formatted at 2026-08-12 07:13:09 on dist-test-slave-csg5"
server_key: "1fa4cd85252b1167d0d0168d26ebc07b"
server_key_iv: "8ac4a56f6bdac9cb75eb19fe9485d517"
server_key_version: "encryptionkey@0"
I20260812 07:13:09.294206 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/wal/instance:
uuid: "b966954119674488a62ab18248303617"
format_stamp: "Formatted at 2026-08-12 07:13:09 on dist-test-slave-csg5"
server_key: "1fa4cd85252b1167d0d0168d26ebc07b"
server_key_iv: "8ac4a56f6bdac9cb75eb19fe9485d517"
server_key_version: "encryptionkey@0"
I20260812 07:13:09.298768 25976 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260812 07:13:09.301671 25994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:09.303530 25976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:13:09.303727 25976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/wal
uuid: "b966954119674488a62ab18248303617"
format_stamp: "Formatted at 2026-08-12 07:13:09 on dist-test-slave-csg5"
server_key: "1fa4cd85252b1167d0d0168d26ebc07b"
server_key_iv: "8ac4a56f6bdac9cb75eb19fe9485d517"
server_key_version: "encryptionkey@0"
I20260812 07:13:09.303887 25976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:09.314795 25976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:09.319018 25976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:09.319275 25976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:09.327553 25976 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.124:34245
I20260812 07:13:09.327878 26054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.124:34245 every 8 connection(s)
I20260812 07:13:09.327883 25969 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } attempt: 3
I20260812 07:13:09.328992 25976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
I20260812 07:13:09.332732 26055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518789, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
I20260812 07:13:09.337620 26055 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:13:09.339411 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 25976
I20260812 07:13:09.339581 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518468557119-32369-0/minicluster-data/master-2/wal/instance
I20260812 07:13:09.339874 32369 external_mini_cluster.cc:1428] Setting key 358ee7af0f013b4dfafa3ca70cc1ea51
I20260812 07:13:09.343856 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.328594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60302)
0812 07:13:09.328750 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.328770 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:09.328908 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:09.331897 (+ 2989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:09.331900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:09.332107 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:09.332227 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.332233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.334056 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.334060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.334765 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.334784 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.335817 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.335820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.335900 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.336722 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:09.336738 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:09.338316 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:09.341354 (+ 3038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.341362 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.341365 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.341707 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.342220 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.342226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.342229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.342345 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.343211 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:09.343215 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:09.343338 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 07:13:09.343464 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 07:13:09.343637 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:13:09.345352 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.329492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60302 (local address 127.31.156.124:34245)
0812 07:13:09.329987 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.330007 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.330052 (+ 45us) server_negotiation.cc:407] Connection header received
0812 07:13:09.331013 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.331047 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.331514 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.331922 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.332381 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.333890 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.334901 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.335646 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.335834 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.338546 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.338552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.338568 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.338603 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.341206 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.341859 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.341868 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.341877 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.342069 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.342459 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.342463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.342464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.343065 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.343224 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.343640 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.344198 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":161,"threads_started":1}
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518789, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:13:09.347808 25883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } attempt: 2
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
I20260812 07:13:09.352756 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.340324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40500 (local address 127.31.156.126:35863)
0812 07:13:09.340487 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.340492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.340932 (+ 440us) server_negotiation.cc:407] Connection header received
0812 07:13:09.341186 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.341190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.341253 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.341336 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.342370 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.343157 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.344757 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.345140 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.345230 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.348170 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.348175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.348178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.348211 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.350483 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.351167 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.351172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.351174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.351266 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.351626 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.351630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.351631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.351851 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.351926 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.352407 (+ 481us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.352550 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:13:09.352739 25969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634: Bootstrap starting.
I20260812 07:13:09.353448 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.340292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:35863 (local address 127.0.0.1:40500)
0812 07:13:09.340788 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.340847 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:09.341035 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:09.341525 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:09.341532 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:09.342050 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:09.342241 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.342248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.343297 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.343302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.344591 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.344602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.345403 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.345406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.345551 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.346222 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:09.346236 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:09.347978 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:09.350607 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.350619 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.350626 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.351038 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.351398 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.351405 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.351407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.351521 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.352023 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:09.352029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:09.352150 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 07:13:09.352988 (+ 838us) client_negotiation.cc:240] Negotiation successful
0812 07:13:09.353225 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":12,"spinlock_wait_cycles":896,"thread_start_us":134,"threads_started":1}
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518788, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
I20260812 07:13:09.354570 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.341159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60310 (local address 127.31.156.124:34245)
0812 07:13:09.341558 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.341575 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.341740 (+ 165us) server_negotiation.cc:407] Connection header received
0812 07:13:09.341887 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.341892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.341940 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.342034 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.343057 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.343960 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.344961 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.345401 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.346003 (+ 602us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.349087 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.349094 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.349098 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.349132 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.351896 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.352479 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.352482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.352484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.352584 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.352963 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.352966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.352968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.353206 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.353409 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.354249 (+ 840us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.354379 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":451,"server-negotiator.queue_time_us":334,"thread_start_us":186,"threads_started":1}
I20260812 07:13:09.355023 26055 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:13:09.357457 25969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:09.358632 25969 log.cc:824] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:09.359277 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.348304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60322)
0812 07:13:09.348484 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.348500 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:09.348583 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:09.349318 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:09.349322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:09.349537 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:09.349674 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.349686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.350791 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.350795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.351895 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.351904 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.352503 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.352510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.352593 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.353099 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:09.353114 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:09.354764 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:09.357549 (+ 2785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.357552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.357554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.357816 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.358146 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.358149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.358151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.358245 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.358752 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:09.358756 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:09.358855 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 07:13:09.358941 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 07:13:09.359074 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 07:13:09.359920 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.348955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60322 (local address 127.31.156.124:34245)
0812 07:13:09.349091 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.349098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.349112 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:13:09.349146 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.349148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.349198 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.349296 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.349787 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.350610 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.352023 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.352377 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.355429 (+ 3052us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.355544 (+ 115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.355547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.355551 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.355576 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.357415 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.357943 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.357947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.357949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.358030 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.358362 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.358365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.358368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.358574 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.359579 (+ 1005us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.359651 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.359766 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:13:09.361485 25969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634: No bootstrap required, opened a new log
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518789, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
I20260812 07:13:09.367029 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.355585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:46199 (local address 127.0.0.1:34624)
0812 07:13:09.356069 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.356084 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:09.356204 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:09.356590 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:09.356593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:09.356848 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:09.356972 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.356979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.358246 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.358249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.359309 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.359690 (+ 381us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.360633 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.360637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.360760 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.361259 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:09.361276 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:09.363148 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:09.365196 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.365200 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.365202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.365422 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.365819 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.365822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.365823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.365923 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.366394 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:09.366397 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:09.366517 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 07:13:09.366637 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 07:13:09.366850 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413}
I20260812 07:13:09.367029 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.355858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34624 (local address 127.31.156.125:46199)
0812 07:13:09.355976 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.355982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.356138 (+ 156us) server_negotiation.cc:407] Connection header received
0812 07:13:09.356321 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.356324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.356376 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.356432 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.357126 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.358039 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.359839 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.360169 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.360253 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.363294 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.363298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.363306 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.363364 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.365048 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.365600 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.365605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.365607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.365718 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.366014 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.366018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.366020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.366275 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.366406 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.366727 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.366856 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 07:13:09.367370 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.355991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40512 (local address 127.31.156.126:35863)
0812 07:13:09.356114 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.356118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.356299 (+ 181us) server_negotiation.cc:407] Connection header received
0812 07:13:09.356493 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.356497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.356562 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.356614 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.357549 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.358277 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.359515 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.359836 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.359911 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.362840 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.362846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.362852 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.362886 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.364890 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.365516 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.365522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.365523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.365626 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.365978 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.365982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.365984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.366283 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.366558 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.367047 (+ 489us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.367206 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 07:13:09.368568 26055 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:13:09.370541 25883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633: Bootstrap starting.
I20260812 07:13:09.373987 25883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633: Neither blocks nor log segments found. Creating new log.
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518789, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
Aug 12 07:13:09 dist-test-slave-csg5 krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
I20260812 07:13:09.375898 25883 log.cc:824] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:09.375239 25969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } }
I20260812 07:13:09.376794 25969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d89e923ef9a46e4857af6998ff4b634, State: Initialized, Role: FOLLOWER
I20260812 07:13:09.377233 25969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } }
I20260812 07:13:09.377961 26064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } } }
I20260812 07:13:09.378218 26064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:09.379177 25969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:09.379714 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.369201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:34245 (local address 127.0.0.1:60324)
0812 07:13:09.369327 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.369343 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:09.369735 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:09.370088 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:09.370091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:09.370356 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:09.370449 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.370455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.371432 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.371436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.372306 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:09.372319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.372879 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.372882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:09.372953 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.373507 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:09.373527 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:09.374918 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:09.377834 (+ 2916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.377837 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.377840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.378119 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.378422 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:09.378425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:09.378427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:09.378519 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:09.379079 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:09.379082 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:09.379193 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 07:13:09.379319 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 07:13:09.379488 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 07:13:09.380457 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.369449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60324 (local address 127.31.156.124:34245)
0812 07:13:09.369558 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.369563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.369576 (+ 13us) server_negotiation.cc:407] Connection header received
0812 07:13:09.369858 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.369861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.369905 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.369972 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.370576 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.371300 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.372454 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.372732 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.372878 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.375079 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.375084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.375087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.375116 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.377667 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.378235 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.378238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.378240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.378322 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.378621 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.378624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.378626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.378935 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.379042 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.380047 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.380237 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 07:13:09.380745 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.369675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.31.156.125:46199)
0812 07:13:09.369803 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.369809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.369824 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:13:09.369965 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.369968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.370010 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.370087 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.371085 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.371691 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.372519 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.372894 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.373012 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.376257 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.376261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.376263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.376312 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.378313 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.378858 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.378862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.378863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.378956 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.379376 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.379380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.379381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.379596 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.379795 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.380450 (+ 655us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.380591 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:13:09.381032 25883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633: No bootstrap required, opened a new log
I20260812 07:13:09.383782 26055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617: Bootstrap starting.
W20260812 07:13:09.385646 26077 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:13:09.385752 26077 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:13:09.386722 26055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:09.387701 26055 log.cc:824] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:09.387755 25883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } }
I20260812 07:13:09.388156 25883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f63d551b072455ba653869528cff633, State: Initialized, Role: FOLLOWER
I20260812 07:13:09.388567 25883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } }
I20260812 07:13:09.390208 25883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:09.391820 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:13:09.393409 25855 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9f63d551b072455ba653869528cff633"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40512
I20260812 07:13:09.393700 25855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 07:13:09.393807 25855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:09.389940 25891 master_runner.cc:186] Error getting master registration for 127.31.156.126:35863: OK, instance_id { permanent_uuid: "9f63d551b072455ba653869528cff633" instance_seqno: 1786518788807102 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:13:09.394073 25891 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:13:09.395062 26055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617: No bootstrap required, opened a new log
I20260812 07:13:09.395259 25855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } }
I20260812 07:13:09.396291 25855 leader_election.cc:288] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [CANDIDATE]: Term 1 election: Requested vote from peers 0d89e923ef9a46e4857af6998ff4b634 (127.31.156.125:46199), b966954119674488a62ab18248303617 (127.31.156.124:34245)
I20260812 07:13:09.396937 25939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f63d551b072455ba653869528cff633" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0d89e923ef9a46e4857af6998ff4b634"
I20260812 07:13:09.397217 25939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:09.399258 25939 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f63d551b072455ba653869528cff633 in term 1.
I20260812 07:13:09.399796 25827 leader_election.cc:302] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d89e923ef9a46e4857af6998ff4b634, 9f63d551b072455ba653869528cff633; no voters:
I20260812 07:13:09.404387 26098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } } }
I20260812 07:13:09.404536 26098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:09.404338 26055 raft_consensus.cc:348] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } }
I20260812 07:13:09.404613 26098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:13:09.404652 26055 raft_consensus.cc:729] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b966954119674488a62ab18248303617, State: Initialized, Role: FOLLOWER
I20260812 07:13:09.404681 26098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [term 1 LEADER]: Becoming Leader. State: Replica: 9f63d551b072455ba653869528cff633, State: Running, Role: LEADER
I20260812 07:13:09.405046 26055 consensus_queue.cc:260] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } }
I20260812 07:13:09.405074 26098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } }
I20260812 07:13:09.407027 26055 sys_catalog.cc:564] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:09.408401 26025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f63d551b072455ba653869528cff633" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b966954119674488a62ab18248303617"
I20260812 07:13:09.408741 26025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:09.408851 26098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f63d551b072455ba653869528cff633. Latest consensus state: current_term: 1 leader_uuid: "9f63d551b072455ba653869528cff633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } } }
I20260812 07:13:09.408979 26098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:09.410480 26025 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f63d551b072455ba653869528cff633 in term 1.
I20260812 07:13:09.404019 25808 master_runner.cc:186] Error getting master registration for 127.31.156.124:34245: OK, instance_id { permanent_uuid: "b966954119674488a62ab18248303617" instance_seqno: 1786518789325186 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:13:09.411320 25808 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 07:13:09.413460 26093 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:13:09.413549 26093 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:13:09.413841 26102 sys_catalog.cc:455] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } } }
I20260812 07:13:09.414193 26102 sys_catalog.cc:458] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:09.419082 26111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:13:09.420310 26111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:13:09.428205 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.400824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40514 (local address 127.31.156.126:35863)
0812 07:13:09.400948 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.400953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.410945 (+ 9992us) server_negotiation.cc:407] Connection header received
0812 07:13:09.411537 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.411541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.411597 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.411653 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.412747 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.413535 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.415529 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.415882 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.415971 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.423962 (+ 7991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.423967 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.423970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.424004 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.425952 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.426519 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.426526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.426531 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.426632 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.427055 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.427061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.427064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.427326 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.427457 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.427856 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.428005 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 07:13:09.428422 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.402150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60336 (local address 127.31.156.124:34245)
0812 07:13:09.402292 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.402299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.409881 (+ 7582us) server_negotiation.cc:407] Connection header received
0812 07:13:09.412103 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.412106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.412164 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.412228 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.413817 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.414495 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.419150 (+ 4655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.419493 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.419600 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.421805 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.421810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.421813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.421845 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.423722 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.424295 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.424299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.424301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.424398 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.424722 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.424726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.424727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.425000 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.425174 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.428130 (+ 2956us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.428274 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 07:13:09.428915 26118 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:13:09.429016 26118 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:13:09.431120 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.401711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34636 (local address 127.31.156.125:46199)
0812 07:13:09.401822 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.401827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.406111 (+ 4284us) server_negotiation.cc:407] Connection header received
0812 07:13:09.407223 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.407226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.407285 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.407340 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.410062 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.410847 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.415801 (+ 4954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.416150 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.416237 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.425634 (+ 9397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.425642 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.425645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.425685 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.427558 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.429519 (+ 1961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.429524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.429528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.429643 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.430120 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.430123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.430125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.430330 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.430496 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.430797 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.430932 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 07:13:09.450631 25939 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [term 1 FOLLOWER]: Refusing update from remote peer 9f63d551b072455ba653869528cff633: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:13:09.450598 26025 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [term 1 FOLLOWER]: Refusing update from remote peer 9f63d551b072455ba653869528cff633: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:13:09.451483 26080 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:13:09.452657 26080 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:13:09.457095 26102 sys_catalog.cc:455] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f63d551b072455ba653869528cff633. Latest consensus state: current_term: 1 leader_uuid: "9f63d551b072455ba653869528cff633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } } }
I20260812 07:13:09.457263 26102 sys_catalog.cc:458] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:09.457755 26064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f63d551b072455ba653869528cff633. Latest consensus state: current_term: 1 leader_uuid: "9f63d551b072455ba653869528cff633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } } }
I20260812 07:13:09.457935 26064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:09.464788 26080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f63d551b072455ba653869528cff633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } } }
I20260812 07:13:09.464948 26080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:09.466431 26102 sys_catalog.cc:455] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f63d551b072455ba653869528cff633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } } }
I20260812 07:13:09.466591 26102 sys_catalog.cc:458] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:09.468364 26064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f63d551b072455ba653869528cff633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } } }
I20260812 07:13:09.468497 26064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:09.467357 26119 mvcc.cc:205] Tried to move back new op lower bound from 7317580961581060096 to 7317580961408831488. Current Snapshot: MvccSnapshot[applied={T|T < 7317580961581060096}]
I20260812 07:13:09.469686 26098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f63d551b072455ba653869528cff633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f63d551b072455ba653869528cff633" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 35863 } } peers { permanent_uuid: "0d89e923ef9a46e4857af6998ff4b634" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 46199 } } peers { permanent_uuid: "b966954119674488a62ab18248303617" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 34245 } } }
I20260812 07:13:09.469810 26098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633 [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:09.470109 26123 mvcc.cc:205] Tried to move back new op lower bound from 7317580961581060096 to 7317580961408831488. Current Snapshot: MvccSnapshot[applied={T|T < 7317580961581060096}]
I20260812 07:13:09.470614 26121 mvcc.cc:205] Tried to move back new op lower bound from 7317580961581060096 to 7317580961408831488. Current Snapshot: MvccSnapshot[applied={T|T < 7317580961581060096}]
I20260812 07:13:09.474692 26111 catalog_manager.cc:1347] Generated new cluster ID: 6db7b66476234b57bc069e4e2fac77f9
I20260812 07:13:09.474766 26111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:13:09.532495 26111 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:13:09.534190 26111 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:13:09.547731 26111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f63d551b072455ba653869528cff633: Generated new TSK 0
I20260812 07:13:09.548957 26111 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:13:09.591291 26057 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.586991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60352 (local address 127.31.156.124:34245)
0812 07:13:09.587172 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.587177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.589693 (+ 2516us) server_negotiation.cc:407] Connection header received
0812 07:13:09.589939 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.589943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.590002 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.590063 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.591095 (+ 1032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60352: BlockingRecv error: recv got EOF from 127.0.0.1:60352 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 07:13:09.593739 25975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.587986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34638 (local address 127.31.156.125:46199)
0812 07:13:09.588127 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.588131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.592307 (+ 4176us) server_negotiation.cc:407] Connection header received
0812 07:13:09.592551 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.592583 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.592650 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.592715 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.593589 (+ 874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34638: BlockingRecv error: recv got EOF from 127.0.0.1:34638 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 07:13:09.595906 25887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.588968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40520 (local address 127.31.156.126:35863)
0812 07:13:09.589108 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.589113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.594597 (+ 5484us) server_negotiation.cc:407] Connection header received
0812 07:13:09.594867 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.594871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.594944 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.595008 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.595733 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40520: BlockingRecv error: recv got EOF from 127.0.0.1:40520 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 07:13:09.601925 25975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.599152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34650 (local address 127.31.156.125:46199)
0812 07:13:09.599297 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.599302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.600686 (+ 1384us) server_negotiation.cc:407] Connection header received
0812 07:13:09.600967 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.600970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.601063 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.601124 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.601772 (+ 648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34650: BlockingRecv error: recv got EOF from 127.0.0.1:34650 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 07:13:09.608382 25975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.606193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34656 (local address 127.31.156.125:46199)
0812 07:13:09.606349 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.606355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.607044 (+ 689us) server_negotiation.cc:407] Connection header received
0812 07:13:09.607281 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.607285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.607347 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.607403 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.608206 (+ 803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34656: BlockingRecv error: recv got EOF from 127.0.0.1:34656 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 07:13:09.615223 25975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.613165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34668 (local address 127.31.156.125:46199)
0812 07:13:09.613317 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.613323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.614005 (+ 682us) server_negotiation.cc:407] Connection header received
0812 07:13:09.614251 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.614255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.614318 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.614382 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.615083 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34668: BlockingRecv error: recv got EOF from 127.0.0.1:34668 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 07:13:09.623980 25975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.622202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34680 (local address 127.31.156.125:46199)
0812 07:13:09.622407 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.622412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.622777 (+ 365us) server_negotiation.cc:407] Connection header received
0812 07:13:09.623030 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.623034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.623096 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.623156 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.623833 (+ 677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34680: BlockingRecv error: recv got EOF from 127.0.0.1:34680 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
W20260812 07:13:09.636897 25975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.634917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.31.156.125:46199)
0812 07:13:09.635053 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.635058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.635697 (+ 639us) server_negotiation.cc:407] Connection header received
0812 07:13:09.635923 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.635927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.635985 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.636055 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.636754 (+ 699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34682: BlockingRecv error: recv got EOF from 127.0.0.1:34682 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 07:13:09.657650 25975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.655787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.31.156.125:46199)
0812 07:13:09.655933 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.655938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.656579 (+ 641us) server_negotiation.cc:407] Connection header received
0812 07:13:09.656801 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.656805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.656862 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.656915 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.657522 (+ 607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34692: BlockingRecv error: recv got EOF from 127.0.0.1:34692 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 07:13:09.694276 25975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.692847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.31.156.125:46199)
0812 07:13:09.693022 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.693027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.693306 (+ 279us) server_negotiation.cc:407] Connection header received
0812 07:13:09.693377 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.693381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.693443 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.693526 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.694139 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34694: BlockingRecv error: recv got EOF from 127.0.0.1:34694 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 07:13:09.762480 25975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.760838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34700 (local address 127.31.156.125:46199)
0812 07:13:09.761033 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.761037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.761257 (+ 220us) server_negotiation.cc:407] Connection header received
0812 07:13:09.761523 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.761526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.761589 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.761701 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.762248 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34700: BlockingRecv error: recv got EOF from 127.0.0.1:34700 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260812 07:13:09.894603 25975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:09.893307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34704 (local address 127.31.156.125:46199)
0812 07:13:09.893518 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.893522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.893605 (+ 83us) server_negotiation.cc:407] Connection header received
0812 07:13:09.893743 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.893746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.893797 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.893872 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.894470 (+ 598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34704: BlockingRecv error: recv got EOF from 127.0.0.1:34704 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
W20260812 07:13:10.154614 25975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:10.153112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.31.156.125:46199)
0812 07:13:10.153327 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:10.153332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:10.153518 (+ 186us) server_negotiation.cc:407] Connection header received
0812 07:13:10.153748 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:10.153752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:10.153816 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:10.153924 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:10.154480 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34710: BlockingRecv error: recv got EOF from 127.0.0.1:34710 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 07:13:10.387308 26077 catalog_manager.cc:1259] Loaded cluster ID: 6db7b66476234b57bc069e4e2fac77f9
I20260812 07:13:10.387425 26077 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634: loading cluster ID for follower catalog manager: success
I20260812 07:13:10.389204 26077 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634: acquiring CA information for follower catalog manager: success
I20260812 07:13:10.390568 26077 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0d89e923ef9a46e4857af6998ff4b634: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:13:10.430629 26118 catalog_manager.cc:1259] Loaded cluster ID: 6db7b66476234b57bc069e4e2fac77f9
I20260812 07:13:10.430764 26118 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617: loading cluster ID for follower catalog manager: success
I20260812 07:13:10.432763 26118 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617: acquiring CA information for follower catalog manager: success
I20260812 07:13:10.434134 26118 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b966954119674488a62ab18248303617: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:13:10.673692 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:10.669063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34714 (local address 127.31.156.125:46199)
0812 07:13:10.669344 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:10.669349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:10.669378 (+ 29us) server_negotiation.cc:407] Connection header received
0812 07:13:10.669511 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:10.669513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:10.669660 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:10.669775 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:10.670102 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:10.670700 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:10.671775 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:10.672035 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:10.672121 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:10.672570 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:10.673105 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:10.673291 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:10.673475 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 07:13:10.673548 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":105,"threads_started":1}
I20260812 07:13:10.682806 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:10.677822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.31.156.124:34245)
0812 07:13:10.678084 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:10.678088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:10.678342 (+ 254us) server_negotiation.cc:407] Connection header received
0812 07:13:10.678570 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:10.678573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:10.678792 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:10.678930 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:10.679235 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:10.679915 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:10.680991 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:10.681264 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:10.681396 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:10.681785 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:10.682272 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:10.682428 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:10.682513 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 07:13:10.682585 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
I20260812 07:13:10.691329 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:10.686489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.31.156.125:46199)
0812 07:13:10.686975 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:10.686981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:10.687379 (+ 398us) server_negotiation.cc:407] Connection header received
0812 07:13:10.687517 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:10.687523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:10.687584 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:10.687668 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:10.688010 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:10.688872 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:10.689947 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:10.690219 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:10.690356 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:10.690719 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:10.690846 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:10.690937 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:10.691061 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 07:13:10.691134 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379}
I20260812 07:13:10.692473 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:10.686560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.31.156.124:34245)
0812 07:13:10.686734 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:10.686741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:10.687034 (+ 293us) server_negotiation.cc:407] Connection header received
0812 07:13:10.687263 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:10.687267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:10.687348 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:10.687413 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:10.687825 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:10.688751 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:10.690465 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:10.690943 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:10.691062 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:10.691917 (+ 855us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:10.692017 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:10.692097 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:10.692284 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 07:13:10.692352 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:13:10.692454 26216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:10.686642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.31.156.126:35863)
0812 07:13:10.687370 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:10.687375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:10.687392 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:13:10.687508 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:10.687511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:10.687701 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:10.687838 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:10.688246 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:10.688902 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:10.690456 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:10.690866 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:10.690977 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:10.691451 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:10.691957 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:10.692068 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:10.692186 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 07:13:10.692277 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":76,"threads_started":1}
I20260812 07:13:10.696579 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 25808
I20260812 07:13:10.708211 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 25891
I20260812 07:13:10.721534 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 25976
2026-08-12T07:13:10Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3508 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:13:10 dist-test-slave-csg5 krb5kdc[26220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:13:10 dist-test-slave-csg5 krb5kdc[26220](info): set up 2 sockets
Aug 12 07:13:10 dist-test-slave-csg5 krb5kdc[26220](info): commencing operation
krb5kdc: starting...
W20260812 07:13:11.898559 32369 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/test-user.keytab.
Aug 12 07:13:11 dist-test-slave-csg5 krb5kdc[26220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:13:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:13:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.156.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
Entry for principal kudu/127.31.156.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab.
I20260812 07:13:12.057507 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:34855
--webserver_interface=127.31.156.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:34855,127.31.156.125:44785,127.31.156.124:32901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:12.187141 26236 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:13:12.187722 26236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:12.187800 26236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:12.187886 26236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:12.192273 26236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:12.192406 26236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:12.192447 26236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:12.192488 26236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:12.192525 26236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:12.198899 26236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:34855,127.31.156.125:44785,127.31.156.124:32901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.126.keytab
--principal=kudu/127.31.156.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.126:34855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.156.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:13:12.200322 26236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:12.201630 26236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:12.213718 26244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:12.213718 26241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:12.213732 26242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:12.214612 26236 server_base.cc:1034] running on GCE node
I20260812 07:13:12.215557 26236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:12.217021 26236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:12.218269 26236 hybrid_clock.cc:648] HybridClock initialized: now 1786518792218238 us; error 53 us; skew 500 ppm
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:12.227662 26236 init.cc:377] Logged in from keytab as kudu/127.31.156.126@KRBTEST.COM (short username kudu)
I20260812 07:13:12.228899 26236 webserver.cc:460] Webserver started at http://127.31.156.126:34301/ using document root <none> and password file <none>
I20260812 07:13:12.229704 26236 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:12.229792 26236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:12.230069 26236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:12.232566 26236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/data/instance:
uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-csg5"
server_key: "acc775d40f059eb91be1f0c19e3540d9"
server_key_iv: "211ee275cd1064992496205d12ff7f37"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.233275 26236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/wal/instance:
uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-csg5"
server_key: "acc775d40f059eb91be1f0c19e3540d9"
server_key_iv: "211ee275cd1064992496205d12ff7f37"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.238184 26236 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:13:12.241448 26251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:12.243187 26236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:13:12.243350 26236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/wal
uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-csg5"
server_key: "acc775d40f059eb91be1f0c19e3540d9"
server_key_iv: "211ee275cd1064992496205d12ff7f37"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.243462 26236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:12.303829 26236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:12.308017 26236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:12.308307 26236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:12.317512 26236 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.126:34855
I20260812 07:13:12.317890 26314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.126:34855 every 8 connection(s)
I20260812 07:13:12.318878 26236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/data/info.pb
I20260812 07:13:12.322290 26316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:12.326789 26316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:13:12.329236 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 26236
I20260812 07:13:12.329387 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-0/wal/instance
I20260812 07:13:12.329723 32369 external_mini_cluster.cc:1428] Setting key 86ed5ffe252fb49331cbdaebb41f6af3
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.156.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.125@KRBTEST.COM" created.
I20260812 07:13:12.344601 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.329429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:34855 (local address 127.0.0.1:37424)
0812 07:13:12.330202 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.330247 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.331272 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.332132 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.332138 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.332681 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:12.332953 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.332968 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.334255 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.334257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.335125 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.335136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.335843 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.335846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.336012 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.336753 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:12.336773 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:12.338396 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:12.340870 (+ 2474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.340881 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.340894 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.341253 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.341692 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.341695 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.341697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.341834 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.342626 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:12.342633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:12.342892 (+ 259us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.343091 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.343686 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":167,"thread_start_us":212,"threads_started":1}
I20260812 07:13:12.344606 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.329429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37424 (local address 127.31.156.126:34855)
0812 07:13:12.330090 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.330121 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.330371 (+ 250us) server_negotiation.cc:407] Connection header received
0812 07:13:12.331675 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.331680 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.331918 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.332143 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.333139 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.334076 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.335254 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.335667 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.335868 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.338572 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.338579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.338592 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.338616 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.340711 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.341426 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.341431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.341432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.341564 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.341982 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.341987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.341989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.342444 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.342647 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.343225 (+ 578us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.343559 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":476,"spinlock_wait_cycles":66432,"thread_start_us":227,"threads_started":1}
I20260812 07:13:12.348456 26316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:13:12.351708 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.351108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:44785 (local address 127.0.0.1:38650)
0812 07:13:12.351434 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.351579 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
W20260812 07:13:12.352085 26254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
I20260812 07:13:12.353715 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.353445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:44785 (local address 127.0.0.1:38652)
0812 07:13:12.353563 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.353620 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 07:13:12.353910 26316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:44785: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab.
Entry for principal kudu/127.31.156.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab.
I20260812 07:13:12.372305 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.125:44785
--webserver_interface=127.31.156.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab
--principal=kudu/127.31.156.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:34855,127.31.156.125:44785,127.31.156.124:32901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:13:12.403333 26316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } attempt: 1
I20260812 07:13:12.404400 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.404002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:44785 (local address 127.0.0.1:38664)
0812 07:13:12.404206 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.404291 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 07:13:12.405292 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.404998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:44785 (local address 127.0.0.1:38674)
0812 07:13:12.405149 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.405213 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 07:13:12.405498 26316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:44785: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
I20260812 07:13:12.473731 26316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } attempt: 2
I20260812 07:13:12.474973 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.474531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:44785 (local address 127.0.0.1:38680)
0812 07:13:12.474790 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.474887 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 07:13:12.475809 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.475567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:44785 (local address 127.0.0.1:38684)
0812 07:13:12.475697 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.475747 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 07:13:12.476082 26316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:44785: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
W20260812 07:13:12.503202 26323 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:13:12.503643 26323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:12.503716 26323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:12.503798 26323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:12.508211 26323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:12.508349 26323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:12.508389 26323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:12.508430 26323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:12.508466 26323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:12.514693 26323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:34855,127.31.156.125:44785,127.31.156.124:32901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.125.keytab
--principal=kudu/127.31.156.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.125:44785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.156.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:13:12.516050 26323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:12.517279 26323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:12.527451 26328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:12.527446 26329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:12.527720 26323 server_base.cc:1034] running on GCE node
W20260812 07:13:12.527460 26331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:12.528714 26323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:12.530025 26323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:12.531360 26323 hybrid_clock.cc:648] HybridClock initialized: now 1786518792531333 us; error 45 us; skew 500 ppm
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:12.534974 26323 init.cc:377] Logged in from keytab as kudu/127.31.156.125@KRBTEST.COM (short username kudu)
I20260812 07:13:12.536322 26323 webserver.cc:460] Webserver started at http://127.31.156.125:35977/ using document root <none> and password file <none>
I20260812 07:13:12.537024 26323 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:12.537143 26323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:12.537431 26323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:12.540066 26323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/data/instance:
uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-csg5"
server_key: "768fd6d289a639b325cc033e75871ad7"
server_key_iv: "71184f60f7e37aa1bc2033dc990651f6"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.540886 26323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/wal/instance:
uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-csg5"
server_key: "768fd6d289a639b325cc033e75871ad7"
server_key_iv: "71184f60f7e37aa1bc2033dc990651f6"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.546633 26323 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 07:13:12.550091 26339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:12.552141 26323 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260812 07:13:12.552330 26323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/wal
uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-csg5"
server_key: "768fd6d289a639b325cc033e75871ad7"
server_key_iv: "71184f60f7e37aa1bc2033dc990651f6"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.552470 26323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:12.567286 26316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } attempt: 3
I20260812 07:13:12.568373 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.567966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:44785 (local address 127.0.0.1:38686)
0812 07:13:12.568168 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.568263 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 07:13:12.569577 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.569126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:44785 (local address 127.0.0.1:38700)
0812 07:13:12.569411 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.569489 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:13:12.569782 26316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.125:44785: Network error: Client connection negotiation failed: client connection to 127.31.156.125:44785: connect: Connection refused (error 111)
I20260812 07:13:12.580068 26323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:12.584354 26323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:12.584599 26323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:12.593581 26323 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.125:44785
I20260812 07:13:12.593910 26394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.125:44785 every 8 connection(s)
I20260812 07:13:12.594911 26323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/data/info.pb
I20260812 07:13:12.598202 26395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:12.602447 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 26323
I20260812 07:13:12.602599 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-1/wal/instance
I20260812 07:13:12.602974 32369 external_mini_cluster.cc:1428] Setting key 5ca5fcf8a38c13990fe629145fad30fd
I20260812 07:13:12.602931 26395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.156.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.156.124@KRBTEST.COM" created.
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:13:12.623440 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.605770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37434 (local address 127.31.156.126:34855)
0812 07:13:12.605972 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.605976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.607218 (+ 1242us) server_negotiation.cc:407] Connection header received
0812 07:13:12.608302 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.608306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.608379 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.608491 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.610043 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.610615 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.611873 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.612126 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.612253 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.618626 (+ 6373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.618630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.618633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.618755 (+ 122us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.620691 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.621469 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.621473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.621477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.621575 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.622003 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.622007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.622009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.622230 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.622351 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.623105 (+ 754us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.623256 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 07:13:12.624423 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.606361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:34855 (local address 127.0.0.1:37434)
0812 07:13:12.607038 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.607088 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.608056 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.608737 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.608748 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.609340 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:12.609881 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.609897 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.610804 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.610808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.611723 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.611733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.612310 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.612314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.612518 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.613308 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:12.613331 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:12.618305 (+ 4974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:12.620916 (+ 2611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.620925 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.620951 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.621339 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.621687 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.621690 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.621692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.621889 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.622371 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:12.622379 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:12.622778 (+ 399us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.623039 (+ 261us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.623447 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":222,"threads_started":1}
I20260812 07:13:12.626586 26395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
I20260812 07:13:12.638626 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.627459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:44785 (local address 127.0.0.1:38702)
0812 07:13:12.627627 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.627642 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.627749 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.628732 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.628735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.628932 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:12.629050 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.629056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.630449 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.630452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.631228 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.631245 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.631994 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.631997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.632094 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.632850 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:12.632863 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:12.634440 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:12.636845 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.636849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.636851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.637064 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.637406 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.637409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.637411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.637497 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.638034 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:12.638037 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:12.638144 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.638262 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.638420 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 07:13:12.638640 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.627665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38702 (local address 127.31.156.125:44785)
0812 07:13:12.628089 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.628099 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.628121 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:13:12.628178 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.628185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.628478 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.628620 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.629198 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.630228 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.631403 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.631848 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.631948 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.634620 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.634633 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.634696 (+ 63us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.634726 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.636693 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.637204 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.637210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.637211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.637292 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.637588 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.637592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.637594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.637906 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.638071 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.638365 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.638505 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":206,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.156.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab.
Entry for principal kudu/127.31.156.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab.
I20260812 07:13:12.640542 32369 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
/tmp/dist-test-taskAFNoqi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.124:32901
--webserver_interface=127.31.156.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab
--principal=kudu/127.31.156.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.156.84:39037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.156.126:34855,127.31.156.125:44785,127.31.156.124:32901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:13:12.641480 26395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:13:12.642614 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.642183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46064)
0812 07:13:12.642388 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.642511 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 07:13:12.642930 26341 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
I20260812 07:13:12.644001 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.643760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46080)
0812 07:13:12.643884 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.643935 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 07:13:12.644642 26395 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:32901: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
I20260812 07:13:12.693982 26395 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } attempt: 1
I20260812 07:13:12.695297 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.694858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46096)
0812 07:13:12.695043 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.695122 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 07:13:12.696352 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.696026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46100)
0812 07:13:12.696157 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.696283 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 07:13:12.696646 26395 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:32901: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
I20260812 07:13:12.713001 26316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } attempt: 4
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
I20260812 07:13:12.725004 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.713955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38714 (local address 127.31.156.125:44785)
0812 07:13:12.714112 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.714119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.714150 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:13:12.714332 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.714335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.714398 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.714499 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.715105 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.715787 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.716648 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.717420 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.717571 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.720534 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.720538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.720540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.720574 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.722775 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.723443 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.723446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.723447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.723552 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.723935 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.723938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.723940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.724196 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.724308 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.724736 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.724871 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 07:13:12.725025 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.713804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:44785 (local address 127.0.0.1:38714)
0812 07:13:12.714065 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.714089 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.714215 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.714528 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.714531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.714837 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:12.714965 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.714972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.715940 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.715944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.716511 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.716522 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.717641 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.717645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.717792 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.718417 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:12.718433 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:12.720291 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:12.722944 (+ 2653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.722949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.722951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.723293 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.723665 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.723668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.723671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.723786 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.724335 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:12.724339 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:12.724490 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.724625 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.724817 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260812 07:13:12.726218 26316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:13:12.727125 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.726826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46102)
0812 07:13:12.726977 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.727050 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 07:13:12.728440 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.728172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46114)
0812 07:13:12.728313 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.728372 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 07:13:12.728636 26316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:32901: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
I20260812 07:13:12.764858 26395 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } attempt: 2
I20260812 07:13:12.765965 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.765536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46122)
0812 07:13:12.765799 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.765872 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260812 07:13:12.766993 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.766647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46124)
0812 07:13:12.766861 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.766919 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260812 07:13:12.767235 26395 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:32901: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
W20260812 07:13:12.776480 26403 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:13:12.776815 26403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:12.776844 26403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:12.776922 26403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:12.781270 26403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:12.781364 26403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:12.781385 26403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:12.781402 26403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:12.781418 26403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:12.787377 26403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.156.84:39037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.156.126:34855,127.31.156.125:44785,127.31.156.124:32901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/krb5kdc/kudu_127.31.156.124.keytab
--principal=kudu/127.31.156.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.156.124:32901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.156.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 06:15:54 UTC on ccycloud.quasar-rnfzce.root.comops.site
build id 4581989
I20260812 07:13:12.787840 26316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } attempt: 1
I20260812 07:13:12.788865 26403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:12.788991 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.788557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46136)
0812 07:13:12.788785 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.788857 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260812 07:13:12.789834 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.789595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46138)
0812 07:13:12.789717 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.789772 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 07:13:12.790055 26316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.156.124:32901: Network error: Client connection negotiation failed: client connection to 127.31.156.124:32901: connect: Connection refused (error 111)
I20260812 07:13:12.790277 26403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:12.800132 26411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:12.800136 26410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:12.800132 26413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:12.800698 26403 server_base.cc:1034] running on GCE node
I20260812 07:13:12.801352 26403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:12.802618 26403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:12.804029 26403 hybrid_clock.cc:648] HybridClock initialized: now 1786518792803942 us; error 95 us; skew 500 ppm
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:12.807723 26403 init.cc:377] Logged in from keytab as kudu/127.31.156.124@KRBTEST.COM (short username kudu)
I20260812 07:13:12.808976 26403 webserver.cc:460] Webserver started at http://127.31.156.124:46373/ using document root <none> and password file <none>
I20260812 07:13:12.809732 26403 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:12.809849 26403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:12.810135 26403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:12.812814 26403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/data/instance:
uuid: "80f38b741b3f48828a04cb47c4554120"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-csg5"
server_key: "460d23a0d5beeb8517a0e4b632b3c92c"
server_key_iv: "9ed921b3ede291551ff5db35a49a6b07"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.813537 26403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/wal/instance:
uuid: "80f38b741b3f48828a04cb47c4554120"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-csg5"
server_key: "460d23a0d5beeb8517a0e4b632b3c92c"
server_key_iv: "9ed921b3ede291551ff5db35a49a6b07"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.818384 26403 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260812 07:13:12.821691 26421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:12.823475 26403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:13:12.823688 26403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/data,/tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/wal
uuid: "80f38b741b3f48828a04cb47c4554120"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-csg5"
server_key: "460d23a0d5beeb8517a0e4b632b3c92c"
server_key_iv: "9ed921b3ede291551ff5db35a49a6b07"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.823907 26403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:12.841487 26403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:12.845495 26403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:12.845746 26403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:12.854377 26403 rpc_server.cc:307] RPC server started. Bound to: 127.31.156.124:32901
I20260812 07:13:12.854723 26482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.156.124:32901 every 8 connection(s)
I20260812 07:13:12.855598 26403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/data/info.pb
I20260812 07:13:12.857270 26316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } attempt: 2
I20260812 07:13:12.858436 26395 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } attempt: 3
I20260812 07:13:12.859472 26483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:12.860755 32369 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu as pid 26403
I20260812 07:13:12.860893 32369 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAFNoqi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518468557119-32369-0/minicluster-data/master-2/wal/instance
I20260812 07:13:12.861210 32369 external_mini_cluster.cc:1428] Setting key 6c27098aff94c1af3d8ace9c1899e306
I20260812 07:13:12.865742 26483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.125@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.126@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
I20260812 07:13:12.876051 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.857913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46142)
0812 07:13:12.858096 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.858114 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.858230 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.861483 (+ 3253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.861487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.861732 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:12.861875 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.861884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.864986 (+ 3102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.864992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.865832 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.865848 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.867064 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.867068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.867163 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.868222 (+ 1059us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:12.868252 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:12.870510 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:12.873520 (+ 3010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.873525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.873528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.873827 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.874349 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.874355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.874358 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.874470 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.875446 (+ 976us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:12.875450 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:12.875557 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.875664 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.875838 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 07:13:12.876425 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.859571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46150)
0812 07:13:12.859737 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.859756 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.859866 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.861482 (+ 1616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.861486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.861732 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:12.861875 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.861884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.864971 (+ 3087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.864976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.865990 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.866008 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.867061 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.867065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.867177 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.868045 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:12.868064 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:12.869883 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:12.873655 (+ 3772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.873659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.873662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.874297 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.874710 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.874714 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.874716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.874824 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.875443 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:12.875446 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:12.875564 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.876032 (+ 468us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.876220 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:13:12.878486 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.863557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46160 (local address 127.31.156.124:32901)
0812 07:13:12.863911 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.863919 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.864892 (+ 973us) server_negotiation.cc:407] Connection header received
0812 07:13:12.864938 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.864943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.864994 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.865076 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.866251 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.867102 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.868488 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.868910 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.869018 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.872350 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.872355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.872359 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.872391 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.874896 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.875436 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.875440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.875441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.875541 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.875917 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.875921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.875922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.876122 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.876328 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.876855 (+ 527us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.877269 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":271,"spinlock_wait_cycles":8832,"thread_start_us":113,"threads_started":1}
I20260812 07:13:12.878490 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.858843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.31.156.124:32901)
0812 07:13:12.859405 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.859424 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.859509 (+ 85us) server_negotiation.cc:407] Connection header received
0812 07:13:12.860530 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.860557 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.861082 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.862142 (+ 1060us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.862847 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.864788 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.866020 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.866892 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.867499 (+ 607us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.870725 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.870729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.870732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.870762 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.873351 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.873978 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.873992 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.873998 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.874199 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.874613 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.874619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.874621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.875307 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.876673 (+ 1366us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.876855 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.877337 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":504,"server-negotiator.queue_time_us":371,"spinlock_wait_cycles":35584,"thread_start_us":161,"threads_started":1}
I20260812 07:13:12.879220 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.859785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46150 (local address 127.31.156.124:32901)
0812 07:13:12.860066 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.860075 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.860090 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:13:12.860530 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.860557 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.861082 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.862142 (+ 1060us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.862847 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.864788 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.866193 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.866896 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.867832 (+ 936us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.870139 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.870147 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.870167 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.870200 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.873504 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.874443 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.874447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.874449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.874552 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.874949 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.874952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.874954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.875305 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.876992 (+ 1687us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.877082 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.877449 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":593,"server-negotiator.queue_time_us":215,"spinlock_wait_cycles":11264,"thread_start_us":82,"threads_started":1}
I20260812 07:13:12.883876 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.868855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37446 (local address 127.31.156.126:34855)
0812 07:13:12.869013 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.869018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.870489 (+ 1471us) server_negotiation.cc:407] Connection header received
0812 07:13:12.870789 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.870796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.870871 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.870934 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.872192 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.873123 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.874636 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.875053 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.875135 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.878332 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.878336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.878339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.878366 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.881154 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.881978 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.881983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.881985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.882113 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.882511 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.882515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.882517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.882889 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.882976 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.883523 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.883684 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:13:12.884779 26316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b: Bootstrap starting.
I20260812 07:13:12.884779 26395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31: Bootstrap starting.
I20260812 07:13:12.884831 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.868191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.126:34855 (local address 127.0.0.1:37446)
0812 07:13:12.870329 (+ 2138us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.870399 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.870601 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.871159 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.871168 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.871787 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:12.872014 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.872026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.873301 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.873306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.874480 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.874491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.875320 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.875324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.875467 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.876089 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:12.876106 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:12.878158 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:12.881287 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.881298 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.881305 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.881750 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.882262 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.882266 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.882268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.882376 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.883076 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:12.883084 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:12.883244 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.884344 (+ 1100us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.884608 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1939,"thread_start_us":184,"threads_started":1}
I20260812 07:13:12.888002 26395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:12.888028 26316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:12.889006 26316 log.cc:824] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:12.888975 26395 log.cc:824] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.126@KRBTEST.COM
I20260812 07:13:12.889772 26483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:13:12.891922 26316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b: No bootstrap required, opened a new log
I20260812 07:13:12.892602 26395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31: No bootstrap required, opened a new log
I20260812 07:13:12.895691 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.882436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.31.156.126:34855)
0812 07:13:12.883236 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.883247 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.883263 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:13:12.883302 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.883305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.883349 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.884258 (+ 909us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.884356 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.885201 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.886373 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.886769 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.887307 (+ 538us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.890407 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.890412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.890414 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.890444 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.892601 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.893234 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.893238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.893240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.893350 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.893697 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.893700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.893702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.893955 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.894151 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.895294 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.895452 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":90,"threads_started":1}
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
I20260812 07:13:12.897161 26316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } }
I20260812 07:13:12.897656 26316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f6dcf61fe6d4e09a4cf3cae77d1882b, State: Initialized, Role: FOLLOWER
I20260812 07:13:12.898152 26316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } }
I20260812 07:13:12.898880 26494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } } }
I20260812 07:13:12.898887 26395 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } }
I20260812 07:13:12.899160 26494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:12.899170 26395 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddcc9ce6a6ab4ac090a3591e0dec9c31, State: Initialized, Role: FOLLOWER
I20260812 07:13:12.899510 26395 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } }
I20260812 07:13:12.900038 26495 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } } }
I20260812 07:13:12.900158 26316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:12.900224 26495 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:12.900876 26395 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:12.901451 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.890412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.125:44785 (local address 127.0.0.1:38730)
0812 07:13:12.890840 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.890853 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.890976 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.891415 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.891418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.891612 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:12.891749 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.891757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.892810 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.892814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.893632 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.893644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.894368 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.894371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.894485 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.895041 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:12.895057 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:12.896689 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:12.899212 (+ 2523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.899216 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.899219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.899536 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.899940 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.899943 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.899946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.900057 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.900703 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:12.900707 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:12.900842 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.900965 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.901163 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349}
I20260812 07:13:12.902386 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.890457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38730 (local address 127.31.156.125:44785)
0812 07:13:12.890580 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.890587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.890930 (+ 343us) server_negotiation.cc:407] Connection header received
0812 07:13:12.891099 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.891105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.891166 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.891261 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.891897 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.892648 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.893777 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.894096 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.894175 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.896884 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.896889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.896892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.896927 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.899035 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.899681 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.899686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.899688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.899813 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.900181 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.900187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.900189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.900514 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.900632 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.902004 (+ 1372us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.902160 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 07:13:12.903497 26483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.156.125@KRBTEST.COM
Aug 12 07:13:12 dist-test-slave-csg5 krb5kdc[26220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.156.124@KRBTEST.COM for kudu/127.31.156.124@KRBTEST.COM
W20260812 07:13:12.911845 26519 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:13:12.912214 26519 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:13:12.909889 26236 master_runner.cc:186] Error getting master registration for 127.31.156.124:32901: OK, instance_id { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" instance_seqno: 1786518792851970 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:13:12.914942 26236 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:13:12.911598 26323 master_runner.cc:186] Error getting master registration for 127.31.156.124:32901: OK, instance_id { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" instance_seqno: 1786518792851970 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:13:12.916852 26323 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:13:12.917049 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.897495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38740 (local address 127.31.156.125:44785)
0812 07:13:12.902740 (+ 5245us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.902745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.902761 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:13:12.902816 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.902821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.902882 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.904243 (+ 1361us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.904347 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.905140 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.906360 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.906777 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.912209 (+ 5432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.912348 (+ 139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.912355 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.912360 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.912381 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.914315 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.915282 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.915286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.915288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.915393 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.915754 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.915760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.915762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.915993 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.916296 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.916450 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.916588 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5172,"thread_start_us":148,"threads_started":1}
I20260812 07:13:12.918120 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.904054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.156.124:32901 (local address 127.0.0.1:46164)
0812 07:13:12.904314 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.904329 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.904409 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.905730 (+ 1321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.905733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.905935 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:12.906046 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.906053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.907037 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.907040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.907768 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.907782 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.908449 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.908452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.908522 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.909030 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:12.909047 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:12.911038 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:12.916259 (+ 5221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.916264 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.916266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.916542 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.916890 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.916895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.916899 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.917009 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.917566 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:12.917569 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:12.917661 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.917780 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.917938 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260812 07:13:12.919116 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.905270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.31.156.124:32901)
0812 07:13:12.905439 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.905444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.905458 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:13:12.905496 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.905499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.905543 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.905618 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.906163 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.906924 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.907911 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.908225 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.908335 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.911186 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.911191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.911193 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.911216 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.916110 (+ 4894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.916663 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.916667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.916668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.916749 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.917133 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.917137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.917139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.917415 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.917528 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.918810 (+ 1282us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.918960 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260812 07:13:12.919814 26516 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:13:12.922782 26516 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:13:12.923213 26483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120: Bootstrap starting.
I20260812 07:13:12.927444 26483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:12.928325 32369 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:13:12.929997 26281 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37462
I20260812 07:13:12.930337 26281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 07:13:12.930383 26483 log.cc:824] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:12.930470 26281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:12.932106 26281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } }
I20260812 07:13:12.933189 26483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120: No bootstrap required, opened a new log
I20260812 07:13:12.933924 26281 leader_election.cc:288] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [CANDIDATE]: Term 1 election: Requested vote from peers ddcc9ce6a6ab4ac090a3591e0dec9c31 (127.31.156.125:44785), 80f38b741b3f48828a04cb47c4554120 (127.31.156.124:32901)
I20260812 07:13:12.933928 26368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31"
I20260812 07:13:12.934235 26368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:12.935853 26368 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f6dcf61fe6d4e09a4cf3cae77d1882b in term 1.
I20260812 07:13:12.936549 26254 leader_election.cc:302] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f6dcf61fe6d4e09a4cf3cae77d1882b, ddcc9ce6a6ab4ac090a3591e0dec9c31; no voters:
I20260812 07:13:12.936946 26494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:13:12.937197 26494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [term 1 LEADER]: Becoming Leader. State: Replica: 7f6dcf61fe6d4e09a4cf3cae77d1882b, State: Running, Role: LEADER
I20260812 07:13:12.937660 26494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } }
I20260812 07:13:12.939095 26483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } }
I20260812 07:13:12.939404 26483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80f38b741b3f48828a04cb47c4554120, State: Initialized, Role: FOLLOWER
I20260812 07:13:12.939790 26483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } }
I20260812 07:13:12.939954 26494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f6dcf61fe6d4e09a4cf3cae77d1882b. Latest consensus state: current_term: 1 leader_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } } }
I20260812 07:13:12.940094 26494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:12.941440 26483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:12.941671 26453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "80f38b741b3f48828a04cb47c4554120"
I20260812 07:13:12.941977 26453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:12.943681 26453 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f6dcf61fe6d4e09a4cf3cae77d1882b in term 1.
I20260812 07:13:12.944473 26535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:13:12.950261 26535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:13:12.960533 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.938372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.31.156.125:44785)
0812 07:13:12.938498 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.938503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.944891 (+ 6388us) server_negotiation.cc:407] Connection header received
0812 07:13:12.946225 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.946230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.946312 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.946373 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.949787 (+ 3414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.950708 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.953523 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.953916 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.953999 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.956291 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.956296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.956299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.956334 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.958289 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.958917 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.958922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.958924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.959026 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.959385 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.959389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.959391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.959666 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.959796 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.960178 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.960334 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:13:12.961905 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.937472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37464 (local address 127.31.156.126:34855)
0812 07:13:12.937604 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.937609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.940894 (+ 3285us) server_negotiation.cc:407] Connection header received
0812 07:13:12.941383 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.941388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.941449 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.941510 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.947490 (+ 5980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.948464 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.952349 (+ 3885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.952781 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.952881 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.956720 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.956725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.956728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.956760 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.959748 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.960311 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.960315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.960317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.960430 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.960753 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.960757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.960759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.961014 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.961167 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.961563 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.961714 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:13:12.963133 26534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } } }
I20260812 07:13:12.963300 26534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:12.968000 26453 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [term 1 FOLLOWER]: Refusing update from remote peer 7f6dcf61fe6d4e09a4cf3cae77d1882b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:13:12.968099 26368 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [term 1 FOLLOWER]: Refusing update from remote peer 7f6dcf61fe6d4e09a4cf3cae77d1882b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:13:12.968876 26494 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:13:12.973776 26529 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 07:13:12.978140 26548 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:13:12.978579 26548 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:13:12.979290 26495 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f6dcf61fe6d4e09a4cf3cae77d1882b. Latest consensus state: current_term: 1 leader_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } } }
I20260812 07:13:12.979454 26495 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:12.979818 26534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f6dcf61fe6d4e09a4cf3cae77d1882b. Latest consensus state: current_term: 1 leader_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } } }
I20260812 07:13:12.980005 26534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:12.981859 26529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } } }
I20260812 07:13:12.982035 26529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:12.982447 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.939801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46178 (local address 127.31.156.124:32901)
0812 07:13:12.939927 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.939932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.962513 (+ 22581us) server_negotiation.cc:407] Connection header received
0812 07:13:12.962579 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.962582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.962645 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.962786 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.965268 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.966158 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.969435 (+ 3277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.969833 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.969919 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.972399 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.972404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.972407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.972444 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.974458 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.980787 (+ 6329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.980792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.980794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.980904 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.981243 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.981259 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.981261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.981528 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.981690 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.982100 (+ 410us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.982265 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:13:12.982787 26549 mvcc.cc:205] Tried to move back new op lower bound from 7317580975987064832 to 7317580975877378048. Current Snapshot: MvccSnapshot[applied={T|T < 7317580975987064832}]
I20260812 07:13:12.983482 26495 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } } }
I20260812 07:13:12.983623 26495 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:12.992386 26494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } } }
I20260812 07:13:12.992544 26494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:12.992784 26535 catalog_manager.cc:1347] Generated new cluster ID: 373cf9dca6dc44998b69a5b89c6e642a
I20260812 07:13:12.992861 26535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:13:12.994283 26534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f6dcf61fe6d4e09a4cf3cae77d1882b" member_type: VOTER last_known_addr { host: "127.31.156.126" port: 34855 } } peers { permanent_uuid: "ddcc9ce6a6ab4ac090a3591e0dec9c31" member_type: VOTER last_known_addr { host: "127.31.156.125" port: 44785 } } peers { permanent_uuid: "80f38b741b3f48828a04cb47c4554120" member_type: VOTER last_known_addr { host: "127.31.156.124" port: 32901 } } }
I20260812 07:13:12.994416 26534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:13:12.994833 26553 mvcc.cc:205] Tried to move back new op lower bound from 7317580975987064832 to 7317580975877378048. Current Snapshot: MvccSnapshot[applied={T|T < 7317580975987064832}]
I20260812 07:13:13.035673 26535 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:13:13.037243 26535 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:13:13.058552 26535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f6dcf61fe6d4e09a4cf3cae77d1882b: Generated new TSK 0
I20260812 07:13:13.059804 26535 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:13:13.098604 26490 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.094568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37476 (local address 127.31.156.126:34855)
0812 07:13:13.094778 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.094783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.097309 (+ 2526us) server_negotiation.cc:407] Connection header received
0812 07:13:13.097560 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.097564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.097624 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.097685 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.098453 (+ 768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37476: BlockingRecv error: recv got EOF from 127.0.0.1:37476 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 07:13:13.102407 26486 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.096555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.31.156.124:32901)
0812 07:13:13.096692 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.096697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.099995 (+ 3298us) server_negotiation.cc:407] Connection header received
0812 07:13:13.101035 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.101039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.101102 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.101198 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.102221 (+ 1023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46188: BlockingRecv error: recv got EOF from 127.0.0.1:46188 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 07:13:13.103678 26402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.095669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38762 (local address 127.31.156.125:44785)
0812 07:13:13.095804 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.095809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.099363 (+ 3554us) server_negotiation.cc:407] Connection header received
0812 07:13:13.100104 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.100108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.100177 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.100243 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.103520 (+ 3277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38762: BlockingRecv error: recv got EOF from 127.0.0.1:38762 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 07:13:13.110179 26402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.107929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.31.156.125:44785)
0812 07:13:13.108062 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.108067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.108868 (+ 801us) server_negotiation.cc:407] Connection header received
0812 07:13:13.109130 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.109135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.109246 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.109309 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.110042 (+ 733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38774: BlockingRecv error: recv got EOF from 127.0.0.1:38774 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 07:13:13.116554 26402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.114394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38776 (local address 127.31.156.125:44785)
0812 07:13:13.114526 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.114531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.115287 (+ 756us) server_negotiation.cc:407] Connection header received
0812 07:13:13.115527 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.115548 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.115619 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.115692 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.116411 (+ 719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38776: BlockingRecv error: recv got EOF from 127.0.0.1:38776 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 07:13:13.123472 26402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.121398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38786 (local address 127.31.156.125:44785)
0812 07:13:13.121532 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.121537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.122180 (+ 643us) server_negotiation.cc:407] Connection header received
0812 07:13:13.122443 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.122447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.122514 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.122574 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.123322 (+ 748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38786: BlockingRecv error: recv got EOF from 127.0.0.1:38786 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 07:13:13.133199 26402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.130852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38796 (local address 127.31.156.125:44785)
0812 07:13:13.131086 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.131091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.131920 (+ 829us) server_negotiation.cc:407] Connection header received
0812 07:13:13.132179 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.132183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.132269 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.132338 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.133053 (+ 715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38796: BlockingRecv error: recv got EOF from 127.0.0.1:38796 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
W20260812 07:13:13.146459 26402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.144562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.31.156.125:44785)
0812 07:13:13.144695 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.144699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.145355 (+ 656us) server_negotiation.cc:407] Connection header received
0812 07:13:13.145577 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.145580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.145641 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.145695 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.146323 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38808: BlockingRecv error: recv got EOF from 127.0.0.1:38808 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 07:13:13.167517 26402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.165519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38820 (local address 127.31.156.125:44785)
0812 07:13:13.165653 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.165658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.166363 (+ 705us) server_negotiation.cc:407] Connection header received
0812 07:13:13.166585 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.166588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.166649 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.166760 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.167391 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38820: BlockingRecv error: recv got EOF from 127.0.0.1:38820 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 07:13:13.204429 26402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.202798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38836 (local address 127.31.156.125:44785)
0812 07:13:13.202988 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.202993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.203222 (+ 229us) server_negotiation.cc:407] Connection header received
0812 07:13:13.203426 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.203430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.203489 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.203573 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.204207 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38836: BlockingRecv error: recv got EOF from 127.0.0.1:38836 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 07:13:13.272244 26402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.270843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38844 (local address 127.31.156.125:44785)
0812 07:13:13.271053 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.271057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.271145 (+ 88us) server_negotiation.cc:407] Connection header received
0812 07:13:13.271291 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.271294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.271349 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.271508 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.272099 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38844: BlockingRecv error: recv got EOF from 127.0.0.1:38844 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 07:13:13.405087 26402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.403451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38846 (local address 127.31.156.125:44785)
0812 07:13:13.403615 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.403619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.403931 (+ 312us) server_negotiation.cc:407] Connection header received
0812 07:13:13.404084 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.404086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.404141 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.404224 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.404910 (+ 686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38846: BlockingRecv error: recv got EOF from 127.0.0.1:38846 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 07:13:13.665336 26402 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:13:13.663931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38850 (local address 127.31.156.125:44785)
0812 07:13:13.664137 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.664144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.664167 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:13:13.664335 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.664341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.664422 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.664530 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.665203 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38850: BlockingRecv error: recv got EOF from 127.0.0.1:38850 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 07:13:13.913913 26519 catalog_manager.cc:1259] Loaded cluster ID: 373cf9dca6dc44998b69a5b89c6e642a
I20260812 07:13:13.914023 26519 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31: loading cluster ID for follower catalog manager: success
I20260812 07:13:13.915796 26519 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31: acquiring CA information for follower catalog manager: success
I20260812 07:13:13.917078 26519 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ddcc9ce6a6ab4ac090a3591e0dec9c31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:13:13.980232 26548 catalog_manager.cc:1259] Loaded cluster ID: 373cf9dca6dc44998b69a5b89c6e642a
I20260812 07:13:13.980384 26548 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120: loading cluster ID for follower catalog manager: success
I20260812 07:13:13.982355 26548 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120: acquiring CA information for follower catalog manager: success
I20260812 07:13:13.984012 26548 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 80f38b741b3f48828a04cb47c4554120: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:13:14.184422 26633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:14.180094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38854 (local address 127.31.156.125:44785)
0812 07:13:14.180365 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:14.180369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:14.180408 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:13:14.180465 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:14.180467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:14.180602 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:14.180697 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:14.181049 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:14.181602 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:14.182695 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:14.182955 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:14.183042 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:14.183386 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:14.183910 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:14.184010 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:14.184136 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 07:13:14.184198 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":104,"threads_started":1}
I20260812 07:13:14.193406 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:14.188450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46200 (local address 127.31.156.124:32901)
0812 07:13:14.188730 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:14.188736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:14.188871 (+ 135us) server_negotiation.cc:407] Connection header received
0812 07:13:14.189240 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:14.189245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:14.189413 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:14.189570 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:14.189857 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:14.190463 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:14.191541 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:14.191820 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:14.191958 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:14.192346 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:14.192885 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:14.193045 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:14.193119 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 07:13:14.193206 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":79,"threads_started":1}
I20260812 07:13:14.202955 26633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:14.198351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38862 (local address 127.31.156.125:44785)
0812 07:13:14.198519 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:14.198524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:14.198542 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:13:14.198753 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:14.198757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:14.198825 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:14.198907 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:14.199331 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:14.199957 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:14.201177 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:14.201679 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:14.201882 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:14.202172 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:14.202285 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:14.202518 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:14.202684 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 07:13:14.202773 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:13:14.203140 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:14.198012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46212 (local address 127.31.156.124:32901)
0812 07:13:14.198148 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:14.198153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:14.198842 (+ 689us) server_negotiation.cc:407] Connection header received
0812 07:13:14.198968 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:14.198971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:14.199030 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:14.199114 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:14.199471 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:14.200422 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:14.201674 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:14.202037 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:14.202152 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:14.202560 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:14.202678 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:14.202745 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:14.202903 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 07:13:14.202981 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:13:14.204527 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:14.198124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37490 (local address 127.31.156.126:34855)
0812 07:13:14.198568 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:14.198575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:14.198591 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:13:14.198767 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:14.198771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:14.198974 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:14.199115 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:14.199423 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:14.200470 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:14.201985 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:14.202699 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:14.202881 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:14.203437 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:14.203846 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:14.203945 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:14.204310 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 07:13:14.204387 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":86,"threads_started":1}
I20260812 07:13:14.208782 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 26236
I20260812 07:13:14.220857 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 26323
I20260812 07:13:14.232761 32369 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAFNoqi/build/debug/bin/kudu with pid 26403
2026-08-12T07:13:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3513 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (7021 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (325665 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST